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ACQUISITION  AND  LOGISTICS 

FOREWORD 


This  Manual  is  issued  under  the  authority  of  DoD  Directive  4245.7,  ‘Transition  from 
Development  to  Production,”  January  19,  1984.  It  provides  assistance  in  structuring 
technically  sound  programs,  assessing  their  risk,  and  identifying  areas  needing  corrective 
action. 

This  Manual  applies  to  the  Office  of  the  Secretary  of  Defense  (OSD),  the  Military  Depart¬ 
ments,  the  Organization  of  the  Joint  Chiefs  of  Staff,  and  the  Defense  Agencies.  The  Term 
“DoD  Components,”  as  used  herein,  refers  to  the  Military  Departments  and  the  Defense 
Agencies. 

This  Manual  is  effective  immediately  and  is  authorized  for  use  by  all  DoD  Components. 
The  guidance  contained  in  this  document  shall  be  used  in  and  tailored  to  individual  acquisition 
programs.  Heads  of  DoD  Components  may  issue  supplementary  instructions,  when 
necessary,  to  provide  the  unique  requirements  within  their  respective  Components.  The 
Commandant,  Defense  Systems  Management  College,  shall  review  the  guidance  set  forth  in 
this  Manual  and  incorporate  the  material  in  College  curricula. 

Send  recommended  changes  to  the  Manual  through  channels  to: 

Director,  Major  Systems  -Acquisition 

Assistant  Secretary  of  Defense  for  Acquisition  and  Logistics 

Room  2A330,  The  Pentagon 

Washington,  D.C.  20301 

DoD  Components  may  obtain  copies  of  this  Manual  through  their  own  publications  chan¬ 
nels.  Other  Federal  agencies  and  the  public  may  obtain  copies  from  the  Department  of 
Defense,  Technical  Information  Center,  Cameron  Station,  Alexandria,  VA  22304,  and  from 
the  U.S.  Department  of  Commerce,  National  Technical  Information  Service,  5285  Port  Royal 
Road,  Springfield,  VA  22161. 


Accesion  For  ( 

NTiS 

DTIC 

Unann< 

Justific 

CRA&I  ’s; 

TAB  C 

aunced  C 

ation 

i 

By 

Distribution  / 

Availability  Codes  ' 

Dist 

A-l 

Avail  r 

Spe 

no  for 

dal 

(Alia., 

James  P.Wade,  Jr, 

Assistant  Secretary  of  Defense 
Acquisition  and  Logistics 


DoD  4245.7-M 


PREFACE 


We  have  been  developing  and  producing  material  with  which  to  defend  this  country  ever  since  our 
independence  over  200  years  ago.  Unfortunately,  we  do  not  handle  the  job  as  well  as  we  should. 
Even  the  programs  we  classify  as  successful  can  be  improved. 

To  my  way  of  thinking,  there  has  been,  is  now,  and  may  always  be  one  principle  in  which  we  must 
strive  for  further  improvement.  That  principle  is  disciplined  engineering.  It  is  of  such  fundamental 
importance  that  it  drives  all  aspects  of  development  and  production  in  any  successful  material 
acquisition.  When  recognjzed,  disciplined  engineering  dictates  all  facets  of  management.  In  short, 
everything  in  the  acquisition  process  in  the  Department  of  Defense  should  be  subservient  to  it;  yet 
most  of  our  management  systems  are  designed  to  circumvent  it  or  excuse  its  omission.  The  irony  is 
that  in  today’s  explosion  of  computer  utilization  and  the  attendant  time  it  takes  to  incorporate 
changes,  we  should  be  seeing  disciplined  engineering  in  all  of  its  grandeur  and  splendor: 
disciplined  engineering  in  design,  disciplined  engineering  in  test,  and  disciplined  engineering  in 
production. 

Additionally,  we  must  strive  for  improvement  in  the  understanding  and  the  timing  of  the  disciplines 
of  design,  test,  and  production.  Successfully  accomplishing  the  engineering  tasks  on  schedule  is 
the  important  “key”  to  reducing  the  risk  of  a  program.  This  has  a  direct  and  profound  impact  on  the 
quality  of  the  decisions  we  make  on  individual  programs,  and,  in  my  judgment,  has  a  more 
immediate  and  potentially  much  greater  return  on  investment  in  time  and  effort  (and  thereby  on  both 
cost  and  performance  as  well).  Most  importantly,  we  can  achieve  this  return  on  investment  with  the 
application  of  current  policy  cited  in  the  parent  document  to  this  Manual  (DoD  Directive  4245.7)  and 
using  established  procedures  within  the  presently  defined  acquisition  process. 

The  key  word  is  discipline!  This  document  is  designed  to  facilitate  that  discipline  that  will  help  us 
collectively  make  wiser  decisions  on  ongoing  programs.  The  term  selected  to  describe  this 
discipline  pretty  well  conveys  its  purpose  and  manner  of  use  in  a  figurative  sense.  The  term  is 
“template.”  We  would  like  to  be  able  to  compare  ongoing  programs  with  these  templates  to  see 
whether  our  decisions  and  the  actions  on  which  they  are  based  fall  within  the  boundaries  of  an 
effective  and  efficient,  low  risk  program. 

I  know  full  well  that  sound  professional  judgment  always  will  be  needed,  and  these  templates  are 
not  offered  as  a  substitute.  I  also  know  that  we  tend  to  repeat  mistakes  in  certain  key  areas  in  the 
acquisition  process  and  that  these  mistakes  are  correctable  largely  by  better  technical  decisions 
well  within  existing  policies  and  established  procedures.  Therefore,  these  templates  are  provided  to 
introduce  discipline  into  the  system,  to  identify  and  give  visibility  to  high  risk  factors,  and  then  to 
provide  the  tools  by  which  risk  can  be  minimized  progressively. 

Accordingly,  I  strongly  commend  this  document  to  you  and  urge  you  to  use  it  diligently. 

O ,  J 

W.  jT^illoughby,  Jr.  j 

Chairman,  1982  Defense  Sciend6  Board 

Task  Force  on  “Transition  from 

Development  to  Production” 
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CHAPTER  1 
INTRODUCTION 


An  often  discussed  aspect  of  the  acquisition  process  in  the 
Department  of  Defense  is  the  length  of  time  it  takes  to  develop 
and  deploy  weapon  systems.  Although  there  have  been 
numerous  attempts  to  shorten  this  cycle,  relatively  little  has 
been  accomplished.  The  cycle  has  grown  longer  and  the 
criticism  stronger. 

The  reasons  for  shortening  the  cycle  are  directed  mainly  toward 
cost,  and  to  some  extent — though  not  enough— toward 
readiness.  However,  in  the  past  few  years,  the  issue  of 
readiness  has  rightfully  gained  visibility  and  importance. 
Although  the  long  acquisition  cycle  certainly  is  not  a  desirable 
situation,  it  might  be  tolerable  if  the  process  yielded  satisfactory 
results.  But  most  new  weapon  systems  are  less  than 
satisfactory  and  require  burdensome  maintenance  and  logistics 
efforts.  Even  with  the  best  of  efforts,  resultant  low  readiness 
often  requires  additional  equipment  in  order  to  meet  the  needs 
of  the  Military  Services.  This  is  due  primarily  to  a  lack  of 
discipline  in  addressing  logistics  requirements  during  design 
and  development. 

In  the  acquisition  process,  first  evidence  of  weapon  system 
problems  sometimes  does  not  become  apparent  until  a 
program  transitions  from  full-scale  development  (FSD)  into 
production.  This  transition  erroneously  is  thought  to  be  a 
discrete  event  in  time.  Most  acquisition  managers  seem  to 
recognize  that  there  is  a  risk  associated  with  the  transition,  but 
perhaps  do  not  know  the  magnitude  nor  the  origin,  because  the 
transition  is  not  a  discrete  event  but  a  process  composed  of 
three  elements:  design,  test,  and  production.  Many  programs 
simply  cannot  succeed  in  production,  despite  the  fact  that 
they’ve  passed  the  required  milestone  reviews.  These 
programs  can’t  succeed  for  technical  reasons,  notwithstanding 
what  is  perceived  as  prior  management  success  related  to  DoD 
acquisition  policy.  A  poorly  designed  product  cannot  be  tested 
efficiently,  produced,  or  deployed.  In  the  test  program  there  will 
be  far  more  failures  than  should  be  expected.  Manufacturing 
problems  will  overwhelm  production  schedules  and  costs.  The 
best  evidence  of  this  is  the  “hidden  factory  syndrome”  with  its 
needlessly  high  redesign  and  rework  costs.  In  addition,  field 
failures  will  destroy  operational  and  training  schedules  and 
increase  costs. 


EFFORTS  TO  SHORTEN 
ACQUISITION  PROCESS 
FAILED 


TRANSITION  FROM 
DEVELOPMENT  TO 
PRODUCTION  IS  THE 
PROBLEM 
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DoD  CORRECTIVE 
MEASURES 
HAVE  FOCUSED  ON 
MANAGEMENT  FIRST 


The  transition  process  is  very  broad  and  it  is  impacted  by 
activities  that  are,  or  more  accurately,  are  not  done  in  the  early 
design  and  test  activities.  For  contractors  who  have  been 
successful  in  designing  and  producing  acceptable  products,  it 
generally  is  recognized  that  the  control  techniques  needed  to 
successfully  complete  the  design,  test,  and  production 
elements  dictate  the  management  system  needed  to  direct  the 
overall  effort.  In  fact,  the  current  management  systems  in 
today’s  industrial  processes  had  their  origins  in  these  design, 
test,  and  production  requirements. 

Corrective  measures  by  the  Department  of  Defense  have 
focused  on  establishing  a  series  of  management  checkpoints 
and  review  activities.  This  becomes  apparent  when  the 
acquisition  process  is  reviewed,  beginning  with  the 
management  perspective  in  DoD  Directive  5000.1  (reference 
(a))  and  DoD  Instruction  5000.2  (reference  (b));  descriptions  of 
the  Defense  Systems  Acquisition  Review  Council  (DSARC)  and 
related  procedures;  and  the  wealth  of  material  that  is  available 
on  the  planning,  programing,  and  budgeting  system  (PPBS) 
and  other  elements  of  defense  planning,  budgeting,  and  funding 
processes.  This  approach  has  been  responsible  for  adding 
numerous  layers  of  management,  and  has  tended  to 
compartmentalize,  matrixize,  and  polarize  the  major  areas  of 
the  acquisition  process:  design,  test,  and  production. 

These  documents  and  the  requirements  that  they  spell  out  are 
important  in  that  they  establish  a  management  grid  that  the 
various  participants  in  the  acquisition  process  must  follow. 
However,  they  do  not  describe  the  industrial  process,  nor  do 
they  provide  intelligence  on  the  management  and  control  of 
those  technical  activities  and  their  related  details  that  can  either 
make  or  break  a  program.  What  has  evolved  as  today’s 
management  system  for  material  acquisition  hardly  recognizes 
the  importance  of  development  and  production,  much  less  does 
it  utilize  the  vast  resources  of  development  and  production  data 
in  any  decision  process.  “Manage  the  fundamentals  of  design, 
test,  and  production  and  the  management  system  will  describe 
itself.”  However,  and  this  is  a  particularly  important  point,  the 
converse  can  never  be  true!  It  is  impossible  to  describe  the 
management  system  first  that  will  take  care  of  the  fundamentals 
of  the  industrial  process — engineering  and  manufacturing. 


This  patently  is  obvious  when  the  management  system  used  by 
the  Department  of  Defense  and  its  Military  Services  is  reviewed. 
Yet,  it  seems  to  be  the  subject  of  continued  and  ongoing 
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interest  at  all  levels  of  both  the  Department  of  Defense  and  the 
Military  Services.  The  central  cry  heard  in  the  halls  of  the 
Pentagon  when  things  go  wrong  is  “reorganize,  restructure  the 
management  system.”  Some  think  that  if  enough  organizational 
boxes  or  enough  people  are  moved,  the  problem  will  go  away. 
Of  course,  it  doesn’t,  yet  those  responsible  for  creating  the 
organizational  mess  think  so.  Consequently,  we  are  left  with  a 
legacy  that  only  grows  worse  with  time.  Why  is  this  the  case? 
Most  probably  because  it  is  the  path  of  least  resistance. 

The  current  review  process,  culminating  in  a  DSARC  decision 
for  major  programs,  has  no  structural  mechanism  that  can 
articulate  with  any  degree  of  certainty  the  risk  associated  with 
the  engineering  and  manufacturing  elements  of  the  weapon 
system  acquisition  process. 

Some  communities  have  suggested  that  the  problem  is  mainly 
one  of  delivering  weapon  systems  that  are  too  complex,  and 
that  reducing  complexity  would  increase  readiness.  However,  a 
recent  Defense  Science  Board  (DSB)  summer  study 
deliberated  the  issue  of  complexity  versus  readiness  and 
concluded  that  although  there  is  a  relationship,  it  is  relatively 
small  and  threat-driven.  It  was  suggested  that  the  probable 
cause  is  inadequate  engineering  and  manufacturing  disciplines 
combined  with  improperly  defined  and  implemented  logistics 
programs.  This  industrial  process  of  weapon  system  acquisition 
demands  a  better  understanding  and  implementation  of  basic 
engineering  and  manufacturing  disciplines.  Once  rigorous, 
disciplined  engineering  practices  are  employed  and 
institutionalized,  both  the  risk  of  deploying  unsuitable  weapon 
systems  and  the  time  in  the  acquisition  cycle  associated  with 
design,  test,  and  production  will  be  reduced. 

Current  DoD  systems  acquisition  policies  do  not  account  for  the 
fact  that  systems  acquisition  is  concerned  basically  and 
primarily  with  an  industrial  process.  Its  structure,  organization, 
and  operation  bear  no  similarity  whatsoever  to  the  systems 
acquisition  process  as  it  is  described  conventionally.  It  is  a 
technical  process  focused  on  the  design,  test,  and  production 
of  a  product.  It  will  either  fail  or  falter  if  these  processes  are  not 
performed  in  a  disciplined  manner,  because  the  design,  test, 
and  production  processes  are  a  continuum  of  interrelated  and 
interdependent  disciplines.  A  failure  to  perform  well  in  one  area 
will  result  in  failure  to  do  well  in  all  areas.  When  this 
happens — as  it  does  all  too  often — a  high  risk  program  results 
whose  equipment  is  deployed  later  and  at  far  greater  cost  than 
planned. 


CAUSES  OF  ACQUISITION 
RISK  ARE  TECHNICAL, 
NOT  MANAGERIAL 
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The  answers  to  these  problems  won’t  be  found  in  another 
revision  of  DoD  Directive  5000.1  (reference  (a))  or  DoD 
Instruction  5000.2  (reference  (b)).  Nor  will  they  be  found  in 
adjustments  to  the  DSARC  or  other  administrative  procedures. 
They  won’t  be  found  in  these  areas,  because  the  problems  are 
technical,  not  managerial. 


DSB  TASK  FORCE 
CORRECTIVE  MEASURES 
FOCUS  ON  TECHNICAL 
SOLUTION 


The  Under  Secretary  of  Defense  for  Research  and  Engineering 
(USDR&E)  recently  has  expressed  more  and  more  concern 
regarding  this  transition  phase.  Consequently,  a  task  force  was 
formed  under  the  auspices  of  the  DSB  to  review  the  various 
subsets  of  the  transition  from  development  to  production.  The 
formal  terms  of  reference  are  summarized  as  follows: 


•  Examine  ways  and  methods  that  will  define  more  clearly 
and  accelerate  the  transition  from  development  into 
production. 

•  Direct  the  inquiry  toward  both  the  producing  industry  and 
the  administering  Government  agency. 

•  Recommend  those  disciplines  and  controls  for 
application  in  those  activities  comprising  design,  test, 

.  and  production  that  result  in  the  timely  delivery  of  a 
quality  product  to  the  operating  forces. 


TEMPLATES  MINIMIZE 
HIGH  TRANSITION  PHASE 
PRODUCT  RISK 


The  major  thrust  of  the  DSB  report  is  directed  toward  the 
identification  and  establishment  of  critical  engineering 
processes  and  their  control  methods.  This  will  lead  to  a  more 
organized  accomplishment  of  these  activities  and  will  place 
more  significance  and  accountability  on  them.  In  order  to  do 
this,  the  task  force  generated  a  matrix  of  the  most  critical  events 
in  the  design,  test,  and  production  elements  of  the  industrial 
process.  These  events  were  then  transformed  into  what  are 
referred  to  as  “templates,”  a  term  that  defines  their  nature  and 
intended  use. 


The  underlying  principle  of  this  approach  is  the  recognition  that 
everyone  in  the  Department  of  Defense  and  all  of  its  contractors 
sincerely  want  to  do  a  good  job.  If  the  proper  environment  exists 
and  the  necessary  tools  to  accomplish  the  work  are  developed, 
satisfactory  products  will  be  forthcoming.  Having  first 
established  these  fundamentals  as  a  reference  point,  it  is  now 
necessary  to  ensure  the  right  environment,  which  in  this  case, 
is  a  matter  of  obtaining  adequate  visibility,  and  establishing  the 
tools,  which  by  their  use  form  a  frame  of  reference  to  evaluate 
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their  proper  application.  In  this  case,  the  tools  are  the 
templates. 

Figure  1-1.  represents  the  DSB  task  force  perspective  of  the 
transition  problem  and  the  action  level  that  must  be  reached  in 
order  to  define  understandable  and  achievable  engineering 
solutions  to  repetitive  transition  risks.  The  key  here  is  to 
recognize  that  risk  is  eliminated  only  when  the  industrial 
process  is  changed,  and  that  change  is  effected  at  a  level  of 
detail  normally  not  visible  to  the  technical  decision  maker. 
Understanding  for  this  crucial  point  is  paramount  to  electing  the 
low  risk  course  of  action. 

The  templates  describe  techniques  for  improving  the 
“acquisition  process”  by  recognizing  it  for  what  it  is— an 
industrial  process  concerned  with  the  design,  test,  and 
production  of  low  risk  products. 


Figure  1-1.  Transition  Problem  Perspective  and  Action  to  Lower  Product  Transition  Risk 
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Selected  areas  of  this  document  stress  the  electrical  and 
electronic  disciplines  because  of  the  significant  role  that  the 
electronics  field  is  playing  in  improving  system  effectiveness 
and  productivity.  Recent  surveys  have  shown  that  the  majority 
of  the  key  technologies  affecting  future  weapon  system 
capability  and  DoD  budgets  are  in  the  electronic  fields.  These 
technologies  include  such  disciplines  as  very  high-speed 
integrated  circuits,  advanced  software  and  algorithms,  machine 
intelligence,  and  space-based  and  short  wave-length  radars. 
However,  emphasis  shall  be  placed  on  maintaining  program 
technical  balance  within  all  disciplines. 

Specific  attributes  override  all  detail  requirements.  These  are 
(1)  assurance  of  design  maturity,  (2)  measurement  of  test 
stability,  and  (3)  certification  of  manufacturing  processes. 
Design  maturity  is  a  qualitative  assessment  of  the 
implementation  of  contractor  design  policy:  Test  stability  is  the 
absence  or  near  absence  of  failures  in  development  testing  of  a 
stable  design.  Certification  of  the  manufacturing  processes 
implies  both  design  for  production  and  proof  of  process  that 
occur  during  pilot  production  (concurrency).  Each  of  the  above 
attributes  is  a  function  of  the  proper  application  of  all  of  the 
templates  identified  in  the  design,  test,  and  production  sections 
of  this  document. 


TEMPLATES  ARE 
BASED  ON  TASK 
FORCE  EXPERIENCE 


The  templates  were  initiated  using  the  reports  of  the  five  panels 
that  made  up  the  DSB  task  force.  The  total  set  of  recommended 
initiatives  and  principles  were  tested  against  their  relationship  to 
“technical  risk,”  using  the  background  and  knowledge  of  the 
members  of  the  task  force  as  the  basis  for  defining  these 
technical  risks  and  for  setting  out  methods  for  minimizing  them 
during  the  transition  from  development  to  production.  From  the 
results,  a  set  of  templates  was  developed  for  use  in  describing 
low  risk  programs.  A  low  risk  program  is  a  program  that  is  not 
likely  to  give  trouble  during  the  transition  out  of  development. 


Each  template  describes  an  area  of  risk  and  then  specifies 
technical  methods  for  reducing  that  risk.  The  templates 
themselves  are  nominally  two-  or  three-page  documents  that 
usually  describe  a  technical  problem  that  in  turn  creates  a  high 
risk  program.  The  templates  then  describe  a  readily  available 
technical  solution  to  the  problem  based  on  the  lessons  learned 
from  analysis  of  a  substantial  number  of  programs. 


Justification  for  the  use  is  then  provided  along  with  supporting 
data. 
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Throughout  this  document  there  are  timelines  for  many 
template  activities  that  begin  and/or  end  between  two  major 
milestones.  In  such  cases,  the  timeline  is  depicted  for  simplicity 
purposes  as  beginning  and/or  ending  in  the  middle  of  the 
program  phase.  It  is  left  to  the  users  of  this  document  to 
determine  how  early  or  how  late  in  the  phase  the  template 
activity  begins  or  ends;  and  such  a  determination  will  be 
influenced  by  the  type  of  program,  the  acquisition  plan,  and  the 
best  judgment  of  experienced  Government  and  industry 
personnel. 

The  subsequent  pages  of  this  document  contain  all  the 
templates  generated  by  the  DSB  task  force  to  reduce  risk 
inherent  in  the  design,  test,  and  production  processes. 
Additional  templates  have  been  generated  as  a  result  of  a  DoD 
and  industrywide  review.  Since  some  risk  is  associated  with 
funding,  facilities,  management  issues,  and  the  transition  plan 
for  design,  test,  and  production,  the  entire  network  of  templates 
is  arranged  in  a  sequence  considered  logical  from  a  typical 
program  manager’s  viewpoint.  Funding  is  presented  first 
because  it  influences  every  other  template  in  the  transition 
document.  The  total  network  of  critical  path  templates  is  shown 
in  figure  1-2. 

In  figure  1-3,  the  time  phasing  associated  with  development  of 
each  of  the  templates  is  identified  as  the  program  progresses 
through  the  material  acquisition  cycle.  Program  risk  is 
introduced  when  a  particular  template  activity  is  started  after  or 
continued  beyond  the  timeline.  For  those  less  familiar  with  the 
DSARC  process  and  its  typical  relationship  with  program 
phasing,  the  conceptual  phase  begins  after  the  justification  for 
major  system  new  start  (JMSNS)  is  approved.  Between 
Milestones  I  and  II,  the  demonstration/validation  phase  occurs 
and  Milestone  II  is  the  beginning  of  FSD.  The  production  phase 
begins  at  Milestone  IIIA  (tooling,  long  lead  time,  and  pilot 
production)  notwithstanding  the  production  preparations  that 
must  begin  early  in  the  FSD  phase,  and  Milestone  NIB  generally 
signifies  the  beginning  of  rate  production. 


TEMPLATE 
APPLICABILITY  IS 
CORRELATED  WITH 
ACQUISITION  PHASES 
AND  MILESTONES 
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Figure  1-2.  Critical  Path  1 
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CHAPTER  2 

INTRODUCTION  FOR  FUNDING  CRITICAL  PATH  TEMPLATE 


Over  the  years,  the  Department  of  Defense  and  the  Military  Services  have  been  struggling 
to  improve  the  acquisition  process.  There  has  been  a  seemingly  endless  proliferation  of 
“blue  ribbon”  panels,  ad  hoc  reviews,  summer  studies,  task  forces,  and  audits,  whose 
memberships  consisted  of  the  most  respected  representatives  of  Government  and 
industry.  Many  of  these  efforts  were  mandated  congressionally,  but  the  increasing 
congressional  focus  (General  Accounting  Office  (GAO)  reports  and  staff  member  inquiries) 
on  DoD  acquisition  programs  indicates  that  Congress  is  not  convinced  that  the  overall 
objective,  namely,  “more  bang  for  the  buck,”  is  being  accomplished. 

There  is  no  doubt  that  past  studies  and  reviews  have  provided  many  practical 
recommendations  and  those  that  were  acted  upon  helped  formulate  current  procedures  for 
the  DSARC  process  and  the  PPBS.  Yet,  there  is  still  concern  whether  the  taxpayer’s 
money  is  being  well  spent  and  whether  our  Armed  Forces  are  being  provided  equipment 
that  works  when  needed.  Why  do  we  have  so  many  cost  overruns  and  why  does  our 
operating  equipment  fail  so  frequently? 

The  answers  are  not  simple.  Some  of  the  more  lofty  answers  pertain  to  the  increasing 
complexity  of  our  hardware,  greater  administrative  reporting  burdens,  changes  in 
administration  policy  from  one  election  to  the  next,  and  variations  in  the  level  of  our 
international  military  commitment  as  it  influences  and  is  influenced  by  the  existing  attitude 
of  the  American  public. 

However,  there  are  at  least  three  answers  that  are  not  so  lofty  and  over  which  we  can  exert 
significant  control.  One  relates  to  the  need  for  more  discipline  in  the  technical  side  of  the 
acquisition  process,  that  is,  more  attention  to  the  engineering  fundamentals  of  design,  test, 
production,  and  supportability;  this  answer  is  the  basic  purpose  of  this  Manual  and  is  well 
described  in  the  Preface  and  Introduction.  A  second  answer  involves  the  critical  resource  of 
personnel  and  is  discussed  in  a  separate  template  in  the  Management  section.  The  third 
answer  is  sound  funding  policy.  In  order  to  avoid  “biting  off  more  than  we  can  chew,”  and 
because  there  are  many  facets  to  funding  policy  concerns,  the  following  template  on  money 
phasing  is  confined  to  research,  development,  test,  and  evaluation  (RDT&E),  and  initial 
production  funding. 
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Inadequate  RDT&E  funding  is,  of  course,  an  obvious  major  risk  area.  Aside  from  this 
“quantity”  issue,  however,  there  are  the  other  funding  risk  areas  that  deal  with  the  phasing 
of  money:  (1)  inadequate  early  RDT&E  funds,  and  (2)  inadequate  early  production  funds 
during  the  latter  phases  of  development  (initial  production  funds  (IPF)  and  long  lead).  Risk 
is  aggravated  by  authorizing  development  without  production  in  mind.  The  development 
decision  is  a  commitment  to  production  that  must  be  supported  by  properly  phased 
funding. 


OUTLINE  FOR  REDUCING  RISK 


•  If  the  all-important  design  and  engineering  effort  is  to  be  funded  adequately,  provide 
a  reasonable  proportion  of  total  RDT&E  funds  in  the  early  years.  Figure  2-1.  is  a 
representation  of  an  idealized  RDT&E  funding  profile. 
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Figure  2-1.  What  We  Should  Oo  (RDT&E  Funding  Profile) 
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Rarely,  however,  are  funds  provided  on  this  type  of  schedule.  Early  dollars  are  hard 
to  find  and  the  profile  shown  in  figure  2-2.  is  a  much  more  typical  situation.  This 
condition  is  aggravated  when  programs  are  started  on  short  notice. 

A  significant  initial  subset  of  this  profile  is  the  RDT&E  funding  spent  on  production 
preparations.  If  this  funding  profile  is  changed,  the  impact  on  transition  must  be 


Figure  2-3.  combines  these  idealized  and  actual  funding  profiles,  and  the  shaded 
area  represents  a  “design  and  engineering  gap”  from  which  the  program  cannot 
recover  by  application  of  later  funds. 

The  first  type  of  funding  risk,  therefore,  can  be  ascertained  by  comparison  of  a 
program’s  funding  profile  with  those  of  figures  2-1 .  and  2-2. 
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Figure  2-3.  The  “Design  and  Engineering”  Gap 
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•  The  second  type  of  risk  reduction  involves  the  early  commitment  of  production 
funds^-while  development  is  still  ongoing— for  tooling,  long  lead  materials,  and 
production  line  startup.  Figure  2-4.  shows  a  graphic  representation  of  the  needed 
buildup  of  production  funds  during  RDT&E  phase  down.  The  “fly  before  buy”  school 
of  acquisition  policy  tends  to  drive  to  the  “too  late”  line.  Excessive  concurrency  can 
result  in  unwise  commitments  indicated  by  the  “too  early”  line.  For  all  programs 
there  will  be  an  optimum  middle  ground  that  results  in  low  RDT&E  risk  and  a 
controlled  “transition  to  production”  (shaded  area). 


Figure  2-4.  Funding  Profiles  (RDT&E  and  Production) 
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Early  availability  of  enough  funding  from  the  RDT&E  and  procurement  appropriations  is 
essential  for  a  smooth  transition  from  development  to  production  and  early  deployment. 
The  proper  focus  must  continue  during  each  annual  budget  cycle.  Without  a  proper  funding 
profile,  it  will  be  impossible  to  keep  the  program  in  technical  balance. 
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CHAPTER  3 

INTRODUCTION  FOR  DESIGN  CRITICAL  PATH  TEMPLATES 


High  risk  of  failure  of  Government  material  acquisition  programs  occurs  at  the  outset  of  the 
design  process.  While  some  level  of  risk  associated  with  a  new  technical  concept  may  be 
unavoidable,  historically  this  risk  has  been  magnified  by  the  misunderstanding  of  the 
industrial  design  disciplines  necessary  to  turn  the  concept  into  a  mature  product.  The 
Government  and  its  contractors  must  share  equal  responsibility  for  this  misunderstanding. 
The  industrial  proposal  and  Government  source  selection  processes  provide  the  last  cost- 
effective  opportunity  to  ensure  application  of  critical  disciplines  during  design  and  therefore 
the  ultimate  achievement  of  design  maturity.  The  application  of  these  disciplines  is  the 
source  of  the  requirement  for  “up  front  funding"  to  minimize  material  acquisition  program 
risk. 

What  is  design  maturity?  It  is  defined  easily  in  the  operational  environment.  A  mature 
design  meets  operational  requirements  without  additional  Government  or  contractor 
intervention — no  further  field  modifications  or  additional  equipment  and  spares  are  required 
to  overcome  design  shortfalls.  In  the  factory,  design  maturity  might  be  indicated  by  the 
tapering  off  of  engineering  change  proposal  (ECP)  traffic,  once  the  test  phase  is  underway, 
if  it  can  be  assumed  that  contract  requirements  are  being  met.  But  what  constitutes  design 
maturity  at  the  conclusion  of  the  design  effort  before  entering  the  formal  test  phase?  This  is 
the  question  faced  at  the  critical  design  review  (CDR),  when  a  decision  to  proceed  with 
fabrication  of  formal  test  articles  must  be  made,  a  decision  on  which  hangs  this  matter  of 
risk. 

Among  the  many  engineering  disciplines  that  must  be  applied  to  arrive  at  a  product  design 
are  several,  bearing  directly  on  risk,  that  have  been  underemphasized  by  the  Government 
and  underutilized  by  its  defense  contractors.  These  disciplines  share  a  common  thread — all 
sen/e  to  reduce  stress  in  the  broadest  sense.  At  the  micro-level,  parts  age  at  a  rate 
dependent  on  the  stress  they  must  endure.  A  design  can  be  said  to  be  mature  when  it 
meets  its  functional  performance  requirements  and  the  applied  stresses  are  well-known, 
and  the  ability  of  every  part  to  endure  those  stresses  can  be  ensured  for  the  required  life  of 
the  product.  The  engineering  disciplines  that  determine  stress  and  ensure  the  ability  of  the 
parts  to  endure  stress  are  those  that  have  received  the  least  attention  in  defense  system 
acquisition. 

The  templates  in  this  section  address  those  neglected  engineering  design  disciplines.  The 
Government  and  its  contractors  bear  equal  responsibility  to  address  these  issues  in  all 
material  acquisition  programs.  The  outlines  for  reducing  risk  will  serve  to  guide  the 
Government  both  in  the  preparation  of  requests  for  proposals  and  in  proposal  evaluation 
during  source  selection.  They  also  will  serve  to  guide  program  managers  in  the  conduct  of 
formal  design  reviews;  and  the  outlines  will  serve  notice  to  Government  contractors  of  the 
unclaimed  risk  issues  on  which  the  Government  intends  to  take  action,  as  a  guide  to. 
ordering  their  internal  policies  and  procedures. 
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Accurate  and  complete  specification  of  the  design  reference  mission  profile  is  required  in 
order  to  support  the  entire  acquisition  process:  design  definition,  stress  analysis,  test 
design,  logistic  support  analysis,  et  al.  The  degree  to  which  the  specified  mission  profile 
corresponds  to  ultimate  service  use  directly  determines  the  degree  of  risk.  Conversely,  this 
degree  of  correspondence  also  affects  progress  toward  design  maturity,  which  is  ultimately 
decided  by  service  use,  not  development  and  operational  testing.  Yet  the  mission  profile  is 
often  left  to  the  contractor’s  discretion,  based  on  a  broad  definition  of  the  Government’s 
intended  use  of  the  product. 


OUTLINE  FOR  REDUCING  RISK 

•  A  functional  mission  profile  is  prepared  that  shows  on  a  time  scale  all  the  functions 
that  must  be  performed  by  the  system  to  accomplish  the  mission.  The  functional 
mission  profile  of  a  system  having  multiple  or  variable  missions  is  defined  by  a 
hypothetical  design  reference  mission  profile  that  contains  a  comprehensive  listing 
of  all  functions  expected  in  every  potential  mission. 

•  An  environmental  mission  profile  is  prepared  that  shows  on  a  time  scale  the 
significant  properties  of  the  surroundings  (and  their  limits)  that  are  likely  to  have  an 
effect  on  the  operation  or  survival  of  the  system.  It  defines  the  total  envelope  of 
environments  in  which  the  weapon  system  must  perform,  including  conditions  of 
storage,  maintenance,  transportation,  and  operational  use. 

•  Mission  functional  and  environmental  profiles  are  prepared  by  the  Government  and 
included  in  requests  for  proposals,  forming  a  basis  for  proposals,  source  selection, 
and  contracts. 
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•  System  functional  and  environmental  profiles  are  prepared  by  the  contractor  on  the 
basis  of  the  total  envelope  of  external  environments  given  by  the  mission  profile,  to 
define  the  functional  requirements  and  induced  environmental  conditions  for  the 
system  and  its  component  parts.  These  become  the  design  requirements  for  the 
component  parts  of  the  system. 

•  The  design  requirements  and  concept  should  include  a  determination  of  support  and 
operability  factors  such  as  the  need  to  interoperate  with  other  Military  Service  and 
allied  systems. 


TIMELINE 


System  functional  and  environmental  profiles  are  prepared  by  the  contractor  during  the 
early  stages  of  concept  development. 
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AREA  OF  RISK 


Design  requirements  are  translated  from  operational  requirements,  stated  by  the  “user” 
activity,  and  frequently  negotiated  or  evolved  during  the  course  of  design.  They  may 
include  design  requirements  that  are  not  measurable  directly  during  the  design  process,  but 
only  can  be  verified  by  extended  formal  tests.  Such  intangible  design  requirements 
are  a  common  cause  of  high  risk. 


OUTLINE  FOR  REDUCING  RISK 

•  Design  requirements  are  developed  in  parallel  with  the  development  of  the  design 
reference  mission  profile.  They  are  defined  completely  in  the  requests  for  proposals, 
in  order  that  one  basis  for  source  selection  may  be  the  offeror’s  approach  to 
satisfying  those  requirements,  including  Government  evaluation  of  corporate  design 
policy  bearing  on  product  risk.  The  complete  design  reference  mission  profile, 
including  support-related  “design  to”  requirements,  is  specified  in  these  design 
requirements. 

•  Primary  design  requirements  are  stated  in  terms  of  parameters  that  can  be 
measured  during  the  design  process,  by  breadboard  testing  or  analogous  design 
action.  Probabilistic  specifications  that  would  require  extended  system  level  testing 
to  verify  compliance  cannot  be  used  by  the  design  engineer  Tor  real  time  design 
decision  making,  and  are  therefore  considered  secondary,  to  be  used  for  planning 
purposes  only. 
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•  When  the  achievement  of  specific  quantitative  system  requirements  is  conditional 
upon  the  performance  of  a  set  of  predefined  tasks,  the  contract  establishes  the 
requirements  for  development  of  approved  program  plans  for  the  accomplishment  of 
these  tasks.  This  will  apply  to  such  disciplines  as  structural  analysis,  weight  control, 
reliability,  maintainability,  systems  safety,  survivability,  corrosion  prevention,  parts 
standardization,  and  similar  activities. 

•  Contractors  are  responsible  for  ensuring  that  subcontractors  and  suppliers  have 
complete  and  definitive  design  requirements  that  flow  down  Government 
requirements  such  as  measurable  parameters  and  performance  of  predefined  tasks. 


TIMELINE 


Design  requirements  are  established  early  in  the  conceptual  phase  and  may  be  altered 
during  validation  as  well  as  increased  in  level  of  detail  and  specificity.  The  design  reference 
mission  profile  influences  the  design  requirements  for  the  component  parts  of  the  system. 
The  contract  for  validation  should  be  structured  to  require  contractor  recommendations  for 
selection  and  tailoring  of  the  optimum  specifications  and  standards  for  application  before 
the  start  of  FSD. 
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AREA  OF  RISK 

Trade  studies  are  essential  elements  of  material  acquisition  programs,  not  only  in  defining 
concepts  that  best  meet  mission  needs,  but  also  in  fine-tuning  selected  concepts  during  the 
design  process.  Concept  validation  may  not  be  complete  at  the  beginning  of  full-scale 
development,  however,  there  js  the  expectation  that  significant  conceptual  problems  can  be 
resolved  during  the  design  process.  In  addition,  reducing  production  risk  frequently  is  not  a 
trade  study  criterion. 


OUTLINE  FOR  REDUCING  RISK 


•  Concepts  representing  new  technology  untested  in  the  production  environment  are 
validated  fully  before  FSD. 

•  Trade  studies  during  the  design  process  are  oriented  towards  reducing  product  risk, 
by  such  means  as  design  simplification,  design  for  compatibility  with  production 
processes,  design  for  ease  of  both  factory  testing  and  built-in  test,  and  design  for 
supportability  and  readiness. 

•  Early  in  the  design  phase,  full  consideration  is  given  to  standard  components  that 
have  been  developed  and  can  meet  the  mission  requirements  (such  as  standard 
avionics,  egress  seats,  etc.). 

•  A  quantitative  trade  parameters  list  is  developed  and  standardized  across  all  design, 
manufacturing,  and  quality  disciplines  as  a  priority  task  early  in  the  RDT&E  program. 
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•  Trade  study  alternatives  are  documented  and  preserved  formally  in  design  review 
documentation  to  ensure  system  engineering  traceability  to  design  characteristics 
downstream. 

•  Production  transition  trade  studies  are  based  on  design  and  performance  criteria  as 
weight  factors  for  trade  study  decisions. 

•  Product  quality  and  reliability  are  not  trade  study  parameters  to  be  sacrificed  for 
cost,  schedule,  or  performance  gains. 

TIMELINE 


A  broad  spectrum  of  trade  studies  is  initiated  during  the  concept  exploration  phase.  These 
trade  studies  continue  on  into  FSD  as  a.logical  approach  to  selecting  the  best  design  once 
the  mission  profile  and  design  requirements  have  been  specified.  The  final  selection  and 
fine  tuning  of  the  design  approach  must  consider  such  factors  as  producibility  and 
operational  suitability  as  well  as  performance,  cost,  and  schedule. 
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AREA  OF  RISK 


The  implementation  of  the  engineering  design  disciplines  involved  in  reducing  product  risk 
is  the  responsibility  of  Government  contractors.  The  existence  or  absence  of  documented 
corporate  policies,  backed  up  by  controlled  engineering  manuals  to  the  necessary  degree 
of  detail,  has  a  direct  bearing  on  the  degree  of  product  risk  associated  with  material 
acquisition.  Many  Government  contractors  do  not  have  such  corporate  policies,  and  when 
these  policies  do  exist,  they  often  lack  implementation  at  the  operating  level,  and  often  lack 
substantive  direction  on  design  for  low  risk. 

OUTLINE  FOR  REDUCING  RISK 

•  Documented  design  policies  and  comprehensive  engineering  documents 
implementing  these  policies  are  visible  and  adhered  to  in  design,  test,  and 
manufacturing  practices. 

— Policies  and  practices  are  sensitive  to  “lessons  learned”  on  past  programs. 

—Abundant  evidence  is  available  that  engineering  practices  are  tailored  to 
product  lines. 

— Policies  and  practices  reflect  the  importance  of  designing  for  supportability  as 
an  integral  part  of  all  design  efforts. 

•  Engineering  design  has  the  documented  responsibility  not  only  for  development  of  a 
low  risk  design  but  also  for  specification  of  test  requirements  and  design  for 
production  and  support. 
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•  Engineering  practices  in  the  form  of  criteria  and  standards  are  included  in  an 
integrated  data  base  accessible  by  design,  test,  production,  and  logistics 
engineering  personnel. 

•  Established  design  review  criteria  are  available  and  are  used  by  an  expert  design 
review  team.  These  criteria,  along  with  specific  means  of  assessing  maturity,  are 
tailored  specifically  to  product  lines. 

•  Design  emphasis  is  placed  on  implementation  of  design  fundamentals,  disciplines, 
and  practices  that  are  known  to  produce  a  low  risk  design  and  that  ensure  design 
maturity  before  design  release. 

TIMELINE 


The  implementation  of  best  practices  in  engineering  design  is  the  responsibility  of 
contractors.  The  existence  or  absence  of  documented  corporate  policy  has  a  direct  bearing 
on  the  degree  of  product  risk  associated  with  material  acquisition.  Appropriate  design 
policies  are  developed  and  proven  before  FSD,  and  they  may  be  updated  and  otherwise 
refined  as  experience  is  gained  during  development. 
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The  design  process  ought  to  reflect  a  sound  design  policy  and  proper  engineering 
disciplines  and  practices— an  integration  of  factors  that  influence  the  production,  operation, 
and  support  of  a  system  throughout  its  life  cycle.  Nevertheless,  concepts  are  often 
selected,  demonstrated,  and  validated  with  little  thought  given  to  the  feasibility  of  producing 
asystem  employing  those  concepts.  This  omission  is  then  carried  forward  into  design,  with 
voids  appearing  in  manufacturing  technology  and  absence  of  proven  manufacturing 
methods  and  processes  to  produce  the  system  within  affordable  cost.  One  of  the  most 
common  sources  of  risk  in  the  transition  from  development  to  production  is  failure  to  design 
for  production.  Some  design  engineers  do  not  consider  in  their  design  the  limitations  in 
manufacturing  personnel  and  processes.  The  predictable  result  is  that  an  apparently 
successful  design,  assembled  by  engineers  and  highly  skilled  model  shop  technicians, 
goes  to  pieces  in  the  factory  environment  when  subjected  to  rate  production.  A  design 
should  not  be  produced  if  it  cannot  survive  rate  production  without  degradation. 


OUTLINE  FOR  REDUCING  RISK 

•  The  potential  to  produce  a  system  is  investigated  carefully  during  the  demonstration 
and  validation  phase  by  means  of  appropriate  producibility  analyses.  Voids  in 
manufacturing  technology  projects  and  manufacturing  methods  and  processes 
peculiar  to  the  design  of  the  specific  system,  subsystems,  and  components  are 
addressed  during  engineering  development.  These  methods  and  processes  are 
proven  by  pilot  lines  and  pilot  quantities,  when  necessary. 

•  The  design  avoids  reliance  on  a  single  unproven  manufacturing  technology  for 
system  critical  performance  characteristics.  Alternative  technologies  and  design 
approaches  are  carried  through  Milestone  II  and  into  engineering  development, 
when  warranted. 
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•  Producibility  engineering  and  planning  is  an  integral  element  of  the  design  process. 
Close  coordination  between  production  and  design  engineering  is  established  from 
the  outset.  Integration  of  life  cycle  factors  in  the  design  is  fostered  by  forming  design 
teams  with  production  engineering  and  support  area  representatives.  Manufacturing 
coordination  is  part  of  production  drawing  release.  Production  engineers  participate 
in  design  concept  development  and  design  engineers  participate  in  production 
planning  to  ensure  design  compatibility  with  production. 

•  The  design  process  specifically  ensures  both  performance  and  producibility 
considerations  for  packaging  of  electronic  components.  Factors  such  as  envelope 
clearances,  package  density,  predicted  versus  actual  weight,  tooling,  and  power 
access  are  equally  as  important  as  component  and  circuit  design  considerations  in 
reducing  transition  and  production  risk. 

•  The  design  is  evaluated  to  ensure  that  the  producibility  and  supportability  factors  are 
being  incorporated.  Producibility  and  supportability  design  changes  are  expedited 
and  incorporated  as  early  as  possible  to  reduce  cost  and  are  not  resisted 
automatically.  These  changes  are  substantiated  promptly  by  necessary  testing. 

•  A  task  analysis  approach,  as  called  out  in  Military  Handbook  46855B  (reference  (c)), 
is  used  to  divide  tasks  among  hardware,  software,  and  operators.  System  design 
then  proceeds  with  this  partitioning  in  mind,  thus  reducing  the  risk  of  complex  tasks 
being  “dumped”  on  operators  when  they  are  better  performed  by  software.  This 
partitioning  also  helps  to  bound  and  define  the  entire  design  effort. 

•  Cross  training  of  engineers  in  design  and  manufacturing  disciplines  actively  is 
supported.  Design  engineers  stay  abreast  of  developments  in  manufacturing 
technology  that  would  affect  the  design. 
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TIMELINE 


The  design  process  describes  all  the  actions  taken  that  culminate  in  a  set  of  drawings 
or  a  data  base  from  which  a  model  can  be  constructed  for  testing  to  verify  specification 
compliance.  Design  criteria  are  developed  and  proven  before  FSD,  and  may  be  updated 
and  otherwise  refined  as  experience  is  gained  during  development.  Production  design 
occurs  concurrently  with  the  other  elements  of  the  design  process.  Much  useful  information 
guidance  technology  on  obtaining  a  producible  design  is  in  Military  Handbook  727 
(reference  (d)). 
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AREA  OF  RISK 


DESIGN’ 

ANALYSIS’ 


Engineering  design  involves  many  specialized  analyses,  most  of  which  are  oriented 
towards  meeting  desired  performance  specifications.  There  also  are  specialized  analyses 
oriented  towards  proofing  design  risk  but  they  are  not  practiced  widely.  When  they  are 
completed,  it  is  often  by  personnel  other  than  the  design  engineers  most  familiar  with  the 
product  design.  These  analyses  are  critical  to  ensuring  a  low  risk  design. 


OUTLINE  FOR  REDUCING  RISK 

•  Stress  and  stress/strength  analyses  are  performed  to  ensure  that  applied  values  of 
all  parameters  specified  in  the  derating,  margin  of  safety,  and  safety  factor  criteria 
for  all  component  parts  and  materials  meet  those  criteria. 

•  “Worst  case”  tolerance  analyses  are  performed  to  ensure  that  the  system  design 
performance  remains  within  specified  limits  for  any  combination  of  component  part 
parameters  within  the  limits  of  their  own  allowable  tolerances. 

•  Sneak  circuit  analyses  are  performed  to  detect  such  unexpected  failure  modes  as 
latent  circuit  paths,  timing  errors,  or  obscure  “cause  and  effect”  relations  that  may 
trigger  unintended  actions  or  block  desired  ones  without  any  part  failures  having 
occurred. 

•  Failure  modes  and  effects  analyses  are  performed  in  order  to  understand  the  effect 
of  each  component  part  failure  on  overall  design  performance,  and  system  and 
equipment  supportability.  Each  component  part  is  analyzed  for  the  purpose  of 
reducing  these  effects  to  a  minimum  through  design  changes. 

•  A  thermal  survey  is  conducted  on  electronic  systems  to  validate  the  accuracy  of  the 
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thermal  stress  analysis,  which  is  then  revised  as  indicated  by  the  survey  to  yield 
more  accurate  results. 

•  Other  analyses  that  may  be  applied  effectively  are  fault  tree,  mass  property,  system 
safety,  maintainability,  life  cycle  cost,  fault  isolation,  redundancy  management,  and 
vibration  survey. 

•  The  results  of  these  analyses  are  used  to  revise  the  design,  as  necessary,  to  reduce 
design  risk,  and  the  analyses  are  updated,  as  necessary,  for  changes  in  design. 
Design  risk  analyses  are  not  performed  simply  for  the  sake  of  meeting  contract  data 
requirements. 

•  CAD  techniques  are  developed  or  acquired,  as  necessary,  to  conduct  these 
analyses  to  the  maximum  extent  possible,  both  as  a  potential  savings  in  engineering 
time  and  cost,  and  in  the  interest  of  improved  and  more  consistent  analytical 
accuracy. 

•  Integrated  logistics  support  analyses  are  performed  to  understand  and  determine 
the  effects  of  a  design  on  supportability  and  logistics  resources  requirements  for  the 
purpose  of  reducing  any  adverse  effects. 

TIMELINE 


Design  analysis  policies  are  developed  and  proven  before  FSD,  but  shall  be  updated  and 
otherwise  refined  as  experience  is  gained  during  development.  Their  use  is  completed 
largely,  except  for  engineering  changes  to  correct  failures,  at  the  conclusion  of  the  design 
process.  _ _ 
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AREA  OF  RISK 


Low  risk  designs  allow  parts  and  materials  to  operate  well  below  their  maximum  allowable 
stress  levels.  Performance-oriented  military  programs  often  attempt  to  use  these  same 
parts  and  materials  at  much  higher  stress  levels.  Pursuit  of  interoperability  and  parts 
standardization  also  may  introduce  similar  risks.  These  choices  often  are  made  by  using 
mathematical  models  and  generic  handbook  data  that  are  imprecise.  The  resultant  high  risk 
may  not  be  discovered  except  by  testing,  often  operational  testing,  which  is  too  late  to  avoid 
extensive  corrective  action. 


OUTLINE  FOR  REDUCING  RISK 

•  The  following  design  criteria  are  used  for  part  operating  temperatures  (except 
semiconductors  and  integrated  circuits).  These  criteria  apply  to  case  and  hotspot 
temperatures. 


<3  watts:  40°C  rise  from  the  part  ambient  with  a  maximum  absolute  temperature  of  +  110°C 

>3  watts:  55°C  rise  from  the  part  ambient  with  a  maximum  absolute  temperature  of  +125°C 

Transformers:  30°C  rise  from  the  part  ambient  with  a  maximum  absolute  temperature  of  +  100°C 
for  MIL-T-27  Class  S  insulation 

Capacitors:  10°C  rise  from  the  part  ambient  with  a  maximum  absolute  temperature  of  +  85°C 
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Of  all  the  forms  of  stress  to  which  electronic  parts  are  susceptible,  thermal  stress  is 
the  most  common  source  of  failures.  The  thermal  stress  guidelines  that  are 
highlighted  have  been  instrumental  in  reducing  the  failure  rate  of  electronic 
equipment  by  up  to  a  factor  of  10  over  traditional  handbook  design  criteria. 

•  The  junction  temperatures  of  semiconductors  and  integrated  circuits  normally  should 
not  exceed  + 1 10°C,  regardless  of  power  rating.  The  failure  rates  of  semiconductors 
decrease  by  as  much  as  a  factor  of  two  for  each  10°C  by  which  their  junction 
temperatures  can  be  lowered.  In  modern  electronic  systems  having  high 
semiconductor  populations,  this  translates  to  an  approximately  equal  decrease  in 
the  overall  system  failure  rate  when  instituted  as  design  policy.  In  one  program 
involving  200  aircraft,  each  5°C  reduction  in  cooling  air  temperature  was  estimated 
to  save  $10  million  in  electronic  system  maintenance  costs  by  reducing  failure  rates. 

•  The  absolute  values  of  operating  temperatures  for  all  electronic  parts  in  a  design  are 
determined  both  by  analysis  and  by  measurement. 

Equipment  used  to  perform  thermal  surveys  on  electronic  systems  and  components 
now  is  available  readily.  This  equipment  usually  is  based  on  infrared  scanning 
techniques,  and  now  is  capable  of  measuring  even  the  junction  temperatures  of 
integrated  circuits  under  development. 

•  Government  contractors  include  in  their  design  policies  and  their  parts  and  materials 
programs  the  derating  criteria  for  all  classes  of  parts  and  materials  to  be  used  in  their 
products,  specifying  absolute  limits  on  all  parameters  to  which  reliability  is  sensitive. 
This  policy  is  subject  to  review  and  approval  by  the  Government  before  contract 
award. 


Stress  derating  practice  ranks  with  mission  profiles  as  the  most  critical  design 
factors  associated  with  low  risk  products. 

•  Program-peculiar  approved  parts  lists  (APL),  in  general  a  sub-set  of  the  Military 
Specification  (MIL-SPEC)  lists,  are  issued  at  the  start  of  FSD.  The  APL  shall  inform 
all  designers  of  the  program’s  standardization  decisions — on  resistors,  capacitors, 
other  electronic  parts,  fasteners,  connectors,  wire,  epoxies,  and  so  forth.  Designers 
must  use  the  selected  standard  parts  when  they  meet  system  requirements  or  justify 
use  of  nonstandard  parts. 
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TIMELINE 


Parts  and  materials  selection  and  stress  derating  policies  must  be  in  place  at  the  start  of 
hardware  development.  The  contractor  design  review  process  is  the  primary  mechanism  to 
ensure  compliance  with  these  policies. 
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AREA  OF  RISK 


SOFTWARE- 

DESIGN 


Many  weapon  systems  now  depend  upon  software  for  their  operation  and  maintenance. 
Whether  the  software  is  embedded  (“tactical”  or  “firmware”)  or  loaded  into  main  memory 
from  peripheral  storage  devices,  the  problems  are  the  same— the  weapon  systems  cannot 
be  qualification  tested  and  they  can’t  function,  in  most  cases,  without  proper  software.  A 
software  error  can  cause  a  weapon  system  failure.  Nevertheless,  software  frequently  fails 
to  receive  the  same  degree  of  discipline  as  hardware  early  in  FSD.  Failure  to  allocate 
system  requirements  clearly  between  hardware  and  software  greatly  increases  the  difficulty 
of  isolating  and  correcting  design  problems.  Industry  experience  shows  that  64  percent  of 
all  software  errors  are  traceable  to  functional  or  logical  design,  with  the  remaining  36 
percent  due  to  coding. 


OUTLINE  FOR  REDUCING  RISK 

•  The  applicability  to  software  in  the  outline  for  reducing  risk  of  every  design  template 
is  considered.  Most  templates  are  as  applicable  to  software  as  to  hardware, 
especially  design  process  and  design  analysis. 

•  Functional  requirements  are  allocated  either  to  hardware  or  to  software,  as 
appropriate,  at  design  start.  These  allocations  usually  are  trade  study  topics,  since  it 
often  is  not  clear  initially  which  functions  should  be  implemented  in  hardware,  and 
which  in  software.  Hardware  and  software  responsibilities  reside  with  one  individual. 


•  Proven  design  policies,  processes,  and  analyses  governing  software  design  are 
employed,  including,  but  not  limited  to  the  following: 

— Rigorous  configuration  control. 

— Chief  programer/designer  teams  and  modular  construction. 

— Structured  programing  and  top-down  design. 
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—  Structured  walkthroughs. 

—  Good  documentation. 

—Traceability  of  all  design  and  programing  steps  back  to  top  level 
requirements. 

—  Independent  review  of  requirements  analyses  and  design  process. 

—  Thorough  test  plan  developed  and  utilized  from  design  start. 

—  Compliance  with  standards. 

—  Structured  flowcharting. 

•  Computer  software  developers  are  accountable  for  their  work  quality,  and  are 
subject  to  both  incentives  and  penalties  during  all  phases  of  the  system  life  cycle. 

•  A  uniform  computer  software  error  data  collection  and  analysis  capability  is 
established  to  provide  insights  into  reliability  problems,  leading  to  clear  definitions 
and  measures  of  computer  software  reliability. 

•  A  software  simulator  is  developed  and  maintained  to  test  and  maintain  software 
before,  during,  and  after  field  testing. 

•  Security  requirements  are  considered  during  the  software  design  process. 

TIMELINE 


It  is  essential  that  software  design  practices  follow  a  disciplined  process  similar  to  proven 
hardware  design  practices.  Design  schedules  for  software  coincide  with  the  hardware 
schedule. 
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AREA  OF  RISK 


COMPUTER-AIDED 
DESIGN  (CAD) 


Many  design  tools  and  analysis  techniques  required  to  achieve  a  mature  design  are  not 
used  or  performed  at  all  because  they  are  time  consuming  and  costly.  Engineers  don’t 
always  follow  the  design  rules  that  their  companies  require.  Producibility  and  testability  of 
the  design  is  often  lacking  due  to  lack  of  communications  with  and  knowledge  of 
manufacturing  processes.  Obtaining  a  good  understanding  of  the  design  before  it  is  built 
and  tested  is  often  lacking,  increasing  the  length  and  cost  of  test  and  fix  periods,  increasing 
cost  of  redesigning  tooling  and  test  equipment,  and  increasing  support  costs  and  the  risk 
during  the  transition  to  production  and  early  deployment.  Obtaining  information  on  part  and 
material  parameter  limitations  and  availability,  as  technology  produces  new  items,  is  time 
consuming  when  available  only  in  printed  form. 


OUTLINE  FOR  REDUCING  RISK 

•  Computer-aided  design  (CAD)  is  carried  out  in  the  factory  as  part  of  a  thorough 
modernization  strategy. 

•  Each  design  engineer  is  provided  the  use  of  an  alphanumeric  computer  terminal. 

•  An  interactive  graphics  terminal  is  provided  for  each  group  of  four  to  six  design 
engineers. 

•  These  graphics  terminals  have  user-friendly  access  to  a  data  base  that  contains  the 
following: 

—  Parts  and  materials  data. 

—  Design  rules  (both  corporate  policy  and  product  specified). 
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—  Design  specifications  (mission  profile,  performance  and  reliability  requirements, 
supportability  design-to  requirements,  limits,  and  boundaries). 

—  Manufacturing  rules  (special  processes,  testability,  and  estimated  quantity). 

—  File  and  retrieve  capability,  including  design  data  and  analysis  results. 

•  Terminals  have  user-friendly  access  to  special  computer  software  (programs)  that 
provide  a  capability  to  accomplish  the  following: 

—  Perform  modeling  and  prototyping. 

—  Perform  simulation  and  performance  analyses. 

—  Perform  special  analyses  such  as  the  following: 

•  Electrical  stress.  •  Failure  modes  and  effects. 

•  Thermal  stress.  •  “Worst  case”  tolerance. 

•  Vibration  stress.  •  Reliability  prediction  and 

•  Sneak  circuit.  allocation. 

—  Maintain  configuration  and  design  release  control. 

—  Help  design  product  tests. 

—  Manage  test  and  failure  analysis  data. 

•  A  common  data  base  is  in  place  to  integrate  CAD  and  computer-aided 
manufacturing  (CAM)  functions  (see  template  on  CAM)  to  achieve  significant  cost, 
schedule,  quality,  supportability,  and  performance  benefits. 

•  An  aggressive  employee  retraining  program  is  in  place  to  provide  for  orderly 
introduction  of  new  skills. 
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TIMELINE 


Through  the  use  of  CAD  equipment,  a  full  complement  of  design  tools  is  available  to 
facilitate  the  design  process  and  satisfy  producibility  objectives. 


A 

j 
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Test  and  inspection  are  integral  functions  of  the  production  and  operational  environment. 
To  survive  the  production  process  without  degradation,  a  design  must  allow  for  access  by 
both  inspectors  and  various  types  of  automatic  testing  approaches. 


OUTLINE  FOR  REDUCING  RISK 

•  Design  criteria  are  provided  for  partitioning,  initialization,  functional  compatibility  with 
automatic  test  equipment  (ATE),  functional  coverage,  modularization,  and  visual  and 
physical  accessibility. 

•  Trade  studies  are  conducted  for  integrated  application  of  built-in  test  (BIT),  ATE,  and 
manual  testing  to  support  fault  detection  and  isolation. 

■  •  Production  design  studies  are  conducted  to  define  inspection,  test,  and  evaluation 
requirements;  to  maximize  inspectability;  and  to  minimize  the  need  for  special 
manufacturing  tests  and  special  factory  or  field  test  equipment. 

•  Classification  of  characteristics  are  noted  on  drawings. 

•  Test  and  evaluation  (T&E)  are  planned  and  coordinated  to  minimize  the  need  for 
subjective  interpretation  of  a  system’s  performance  design  requirements. 

•  Factory  test  consumes  no  more  than  10  percent  of  expected  product  life. 

•  System  level  functional  testing  is  conducted  at  a  level  that  meets  but  does  not 
exceed  operational  use  requirements. 
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TIMELINE 


To  provide  for  efficient  and  economical  manufacture,  consideration  must  be  given  to 
providing  the  proper  test  and  inspection  capabilities  in  the  basic  equipment  design.  Policies 
governing  design  for  testing  are  established  before  FSD,  and  such  design  is  completed 
largely  at  the  conclusion  of  the  design  process. 
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BUILT-IN  TEST 


Built-in  test  (BIT)  circuitry  offers  not  only  ease  of  maintenance  in  the  field  but  also  more 
rapid  troubleshooting  during  factory  test  and  production.  Many  designs  do  not  include 
sufficient  BIT  capability  to  isolate  failures  to  the  single  faulty  line-replaceable  or  weapon- 
replaceable  assembly,  much  less  the  shop-replaceable  assembly  or  component  part.  One 
of  the  more  common  results  is  the  line  removal  of  functional  assemblies  along  with  the 
nonfunctional  one,  increasing  downtime  and  causing  unnecessary  backlogs  in  logistic 
support.  The  argument  is  heard  frequently  that  additional  BIT  equipment  itself  adds  to 
product  risk  beyond  the  value  it  might  have  in  maintenance.  This  argument  may  have  had 
validity  in  an  earlier  era,  but  not  with  today’s  complex  yet  low  risk  integrated  circuitry. 


OUTLINE  FOR  REDUCING  RISK 

•  Maintenance  and  support  requirements  are  defined  before  initiation  of  BIT  design. 

•  Design  criteria  are  provided  for  the  contribution  of  BIT  circuitry  to  product  risk, 
weight,  volume,  and  power  consumption.  These  criteria  are  established  by 
Milestone  II. 

•  Trade  studies  are  conducted  for  each  maintenance  level  on  the  interaction  of  BIT, 
automatic  test  equipment,  and  manual  test  in  support  of  fault  detection  and  isolation; 
and  to  optimize  BIT  allocation  in  hardware,  software,  and  firmware. 

•  Production  design  studies  are  conducted  to  define  the  use  of  BIT  in  manufacturing 
inspection,  test,  and  evaluation. 

•  BIT  criteria,  at  a  minimum,  detect  all  mission  compromising  failures,  and  validate  all 
redundant  functions. 
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TIMELINE 


BIT  is  a  significant  factor  in  the  initial  design  planning  and  tradeoff  analyses  and  must  be 
evaluated  in  subsequent  design  reviews.  Concepts  for  BIT  that  are  validated  during  the 
normal  program  validation  phase  may  be  adopted  for  the  final  design.  BIT  design  is 
completed  and  validated  during  full-scale  development. 
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A  common  source  of  risk  in  the  transition  from  development  into  production  is  failure  to 
establish  and  maintain  a  strong  configuration  control  system.  Direct  application  of 
boilerplate  policies  and/or  invoking  MIL-SPECs  leads  to  ineffective  control  or  overly 
complex  and  costly  approaches  to  managing  configuration.  In  a  loosely  implemented 
control  system,  design  changes  can  occur  without  proper  maintenance  of  the  configuration 
change  documentation  after  the  design  freeze  is  established.  Lack  of  a  good  configuration 
control  system  leads  to  many  pitfalls,  including  an  unknown  design  baseline,  excessive 
production  rework,  poor  spares  effort,  stock  purging  rather  than  stock  control,  and  an 
inability  to  resolve  field  problems.  Poor  configuration  control  is  a  leading  cause  of 
increasing  program  costs  and  lengthening  procurement  schedules. 


OUTLINE  FOR  REDUCING  RISK 

•  An  effective  configuration  control  system  contains  the  following  features: 

—  It  is  tailored  from  an  effective  set  of  guidelines  and  standards  to  fit  the  nature  of 
the  program  including  hardware  and  logistics  support  elements. 

—  Corporate  or  division  policy  recognizes  the  importance  of  proper  configuration 
management  in  the  development  of  a  new  program,  and  emphasizes  the  need 
to  generate  an  adequate  plan  for  implementation. 

—A  configuration  management  plan  is  streamlined,  yet  adequately  encompasses 
the  entire  life  cycle  of  the  program,  recognizing  the  requirements  of  each  phase 
of  the  life  cycle  and  the  complexity  of  the  system  configuration. 
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—The  configuration  management  plan  establishes  the  mode  of  operation  and 
interface  relationships  among  vendors,  subcontractors,  contractor,  and 
customer. 

•  Proper  staffing  and  authority  commensurate  with  responsibility  are  essential  to  the 
success  of  a  configuration  management  organization. 

•  The  specification  tree,  engineering  release,  and  drawing  discipline  are  managed  by 
documentation  requirements  that  have  been  established  through  the  configuration 
management  plan. 

•  Training  in  the  established  configuration  management  system  is  essential  for  a 
smooth  configuration  management  program. 

•  A  sound  configuration  management  system  recognizes  that  strict  discipline  is 
necessary  to  organize  and  implement,  in  a  systematic  fashion,  the  process  of 
documenting  and  controlling  configuration. 

•  Dynamic  change  control  boards  and  status  accounting  systems  that  are  updated 
frequently  by  timely  feedback  from  user  activities  are  indicative  of  effective 
configuration  management. 

•  Good  configuration  control  procedures  ensure  the  establishment  and  maintenance 
of  design  integrity. 

•  Configuration  audits  are  performed  to  establish  the  design  baseline  and  to  validate 
the  drawing  package  before  production  release. 

•  Manufacturing  engineering  interfaces  with  configuration  control  of  work  instruction 
planning. 

•  The  transition  from  contractor  to  Government  responsibility  is  made  when  the  design 
is  largely  mature  and  when  field  support  will  be  enhanced. 
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TIMELINE 


The  application  of  configuration  control  on  a  program  is  essential.  For  effective  utilization,  it 
should  be  tailored  to  fit  the  nature  of  the  program.  Configuration  control  policies  are 
established  early  in  the  development  and  the  design  baseline  configuration  is  stabilized 
before  production. 
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TEMPLATE 


AREA  OF  RISK 


While  defense  contracts  usually  require  formal  design  reviews,  they  often  lack  specific 
direction  and  discipline  in  the  design  review  requirement,  resulting  in  an  unstructured 
review  process  that  fails  to  fulfill  either  of  the  two  main  purposes  of  design  review,  which, 
are:  (1)  to  bring  to  bear  additional  knowledge  to  the  design  process  to  augment  the  basic 
program  design  and  analytical  activity;  and  (2)  to  challenge  the  satisfactory 
accomplishment  of  specified  design  and  analytical  tasks  needed  for  approval  to  proceed 
with  the  next  step  in  the  material  acquisition  process. 


OUTLINE  FOR  REDUCING  RISK 

•  The  Government  and  its  contractors  recognize  that  design  reviews  represent  the 
“front  line”  where  readiness  for  transition  from  development  to  production  is  decided 
ultimately.  Design  review  policy,  schedule,  budget,  agenda,  participants,  actions, 
and  followup  are  decided  in  view  of  this  foremost  need. 

•  Design  reviews  are  included  in  all  material  acquisition  programs  in  accordance  with 
existing  Government  requirements.  A  design  review  plan  is  developed  by  the 
contractor  and  approved  by  the  Government.  The  design  review  plan  provides  for 

— both  Government  design  reviews  and  internal  contractor  design  reviews  and 
inspections. 

•  Design  review  requirements  flow  down  to  subcontractors  and  suppliers  to  ensure 
proper  subcontractor  internal  design  review  practices  and  to  provide  timely 
opportunities  for  both  the  contractor  and  the  Government  to  challenge 
subcontracted  material  design. 
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•  Government  and  contractor  design  review  participants  are  selected  or  recruited  from 
outside  the  program  to  be  reviewed,  on  the  basis  of  experience  and  expertise  in 
challenging  the  design,  and  have  a  collective  technical  competence  greater  than  or 
equal  to  that  of  the  designers  responsible  for  the  design  under  review. 

•  Manufacturing,  product  assurance,  and  logistics  engineering  functions  are 
represented  and  have  authority  equal  to  engineering  in  challenging  design  maturity. 

•  Design  reviews  use  computer-aided  design  analyses,  whenever  available,  and 
include  review  of  production  tooling  required  at  the  specific  program  milestone. 


TIMELINE 


Design  reviews  must  be  performed  by  technically  competent  personnel  in  order  to  review 
design  analysis  results  and  design  maturity,  and  to  assess  the  technical  risk  of  proceeding 
to  the  next  phase  of  the  development  process.  Design  review  policies  are  established 
before  FSD,  and  the  design  reviews  are  completed  by  the  conclusion  of  FSD. 
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DESIGN:' 

RELEASE 


One  of  the  most  critical  concerns  in  the  transition  from  development  to  production  is  the  risk 
associated  with  the  timing  of  design  release.  On  many  programs,  design  release  schedules 
are  established  by  “back  planning”  from  manufacturing  schedules  or  ambitious  marketing 
considerations .  As  a  result,  the  design  engineer  is  expected  to  meet  unrealistic  milestones 
forcing  him  or  her  to  deviate  from  standard  design  practices.  The  results  are  predictable: 
design  solutions  are  not  the  most  beneficial  to  the  overall  design,  interface  considerations 
are  glossed  over,  costly  redesigns  occur,  and  necessary  documentation  is  sketchy. 
Expedited  and  advanced  design  releases  generally  create  the  need  for  second  and  third 
generation  effort.  On  the  other  extreme,  when  a  design  release  is  scheduled  beyond  the 
normal  period  required  to  complete  the  design,  the  designer  is  tempted  to  add  undue 
complexity  to  the  basic  design  rather  than  improve  inherent  reliability  or  maintainability  or 
reduce  costs. 


OUTLINE  FOR  REDUCING  RISK 

•  Documented  corporate  policy  clearly  identifies  practices  and  procedures  for  design 
drawing  releases  to  facilitate  transition  and  reduce  production  risk. 

•  The  design  release  disciplines  practiced  by  the  contractor  are  flowed  down  to 
subcontractors  and  suppliers. 

•  By  applying  uniform  practices  and  procedures  dealing  with  technical  requirements 
and  evaluating  current  manufacturing  capability,  realistic  design  release  dates  can 
be  established. 

•  In  areas  of  high  manufacturing  risk,  alternate  design  approaches  are  planned  and 
evaluated  to  ensure  that  the  design  release  schedule  is  maintained. 
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•  Complex  designs  are  validated  before  design  release  by  fabricating  preproduction 
manufacturing  models  and  feeding  results  back  to  design  for  corrective  action.  This 
step  increases  the  assurance  that  the  design  release  documentation  will  support  full- 
scale  production. 

•  The  design  release  documentation  includes  all  necessary  information  required  for 
an  orderly  transition  from  design  to  production. 

•  A  formal  review  of  the  design  release  documentation  is  conducted  at  the  critical 
design  review  (CDR). 

•  The  design  baseline  is  established  and  validated  as  part  of  the  design  release. 

•  All  design-related  testing,  including  qualification  testing,  is  completed  before  design 
release,  to  ensure  that  the  design  has  reached  acceptable  maturity. 


TIMELINE 


Integral  to  the  development  process  are  the  facts  that  at  some  point,  creative  design  must 
then  be  released  to  manufacturing.  Design  release  is  completed  with  the  acceptance  of  the 
design  through  the  CDR  and  qualification  test  process. 
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CHAPTER  4 

INTRODUCTION  FOR  TEST  CRITICAL  PATH  TEMPLATES 


During  the  development  cycle  of  a  weapon  system  various  tests  are  performed  by 
subcontractors,  prime  contractors,  and  the  Government.  In  the  early  stages  of 
development,  these  tests  are  used  in  evaluating  design  approaches  and  selecting  design 
solutions  for  further  development.  As  the  design  matures,  the  tests  become  more  complex 
in  attempting  to  provide  confidence  that  the  weapon  system  will  perform  satisfactorily  in  the 
actual  operational  environment. 

As  weapon  systems  have  become  more  sophisticated,  test  requirements  have  been  added 
with  little  consideration  being  given  to  possible  duplication  of  effort  or  the  elimination  of 
older  tests  that  no  longer  are  needed.  Attempts  also  have  been  made  to  “standardize”  test 
environments.  In  many  instances,  these  “standard”  environments  have  shown  little  relation 
to  the  actual  operational  environment,  resulting  in  costly  engineering  changes  to  weapon 
systems,  after  initiation  of  production  and  deployment,  in  order  to  correct  basic  design 
deficiencies  that  would  have  been  detected- before  production  had  a  proper  environment 
been  used. 

The  DSB  task  force  reviewed  the  test  and  evaluation  experience  of  several  major  DoD 
programs  and  the  contributions  of  the  test  programs  towards  reducing  the  risk  of  transition 
from  development  to  production.  Areas  investigated  included  topics  such  as  integrated  test 
plans;  operational  test  environments;  reliability  development  tests;  reliability  demonstration 
tests;  software  tests;  Government  participation  in  full-scale  engineering  development  tests; 
initial  operational  test  and  evaluation;  application  of  the  test,  analyze,  and  fix  (TAAF) 
philosophy  during  transition;  and  the  feedback  of  information  from  initial  field  use  of 
production  weapon  systems. 

The  issues  and  guidelines  provided  in  this  section  represent  the  most  significant  areas 
requiring  special  management  attention  in  order  to  reduce  the  risk  of  transition  from 
development  to  production.  The  process  to  integrate  and  document  test  requirements  for 
the  end  item  begins  with  the  preparation  and  generation  of  the  test  and  evaluation  master 
plan  (TEMP). 
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AREA  OF  RISK 

Although  every  development  program  has  a  defined  test  plan,  this  plan  usually  specifies  a 
series  of  standard  tests  that  have  not  been  integrated  properly.  Integration  includes  the 
careful  accounting  of  objectives,  environments,  test  article  configurations,  data 
requirements,  and  schedules.  Recognizing  that  T&E  is  a  major  cost  driver,  the  objectives  of 
test  integration  are  to  minimize  overlaps  and  gaps,  to  collect  maximum  intelligence  from 
every  test,  and  to  ensure  a  smooth  and  effective  test  program  at  minimum  cost.  The 
absence  of  a  carefully  integrated  test  plan  is  a  certain  indicator  of  a  high  risk  program. 

Critical  parameters  and  characteristics  measured  in  production  acceptance  tests  (PATs)  do 
not  give  a  sufficiently  high  level  of  confidence  that  the  product  meets  its  specification. 
Production  configuration  changes  introduced  without  recertifying  the  validity  of  the  PAT 
further  increase  product  risk. 


OUTLINE  FOR  REDUCING  RISK 

•  Early  in  the  program  initiation  phase  an  integrated  test  plan  (ITP)  is  prepared  by  the 
prime  contractor  for  Government  approval  that  maximizes  efficiency  in  testing,  as 
follows: 

—  Includes  all  development  and  qualification  tests  (prime  contractor, 
subcontractors,  and  Government)  at  the  system  and  subsystem  levels. 


—  Identifies  duplicate  test  activities  and  missing  test  activities. 
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—  Provides  for  the  most  efficient  use  of  test  facilities  and  test  resources. 

•  This  ITP  is  updated  periodically. 

•  Government  participation  in  contractor  testing  of  weapon  systems  includes 
operating  the  system  a  portion  of  the  time  during  FSD. 

•  Initial  operational  test  and  evaluation  (IOT&E)  is  conducted  during  the  transition  from 
development  to  production,  using  the  latest  available  configuration,  when  possible. 

•  Qualification  test  articles  are  representative  of  production  units. 

•  Production  acceptance  testing  is  conducted  on  all  production  items,  to  ensure  the 
continuing  effectiveness  of  the  manufacturing  processes,  equipment,  and 
procedures.  This  includes  revalidation  of  acceptance  test  procedures  when 
significant  changes  occur  in  the  configuration  or  the  production  processes. 

•  Ensure  that  test  tolerances  are  tunneled  from  component  (most  restrictive)  to 
system  (least  restrictive)  within  system  specification  performance  parameters. 

•  Reasonable  probability  that  the  product  meets  previously  qualified  performance 
requirements  is  demonstrated  by  the  production  acceptance  test,  in  terms  of  both 
thoroughness  and  severity,  as  a  prerequisite  to  product  acceptance  by  the 
Government. 

•  Figure  4-1 .  shows  the  essential  elements  of  an  ITP. 
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To  ensure  that  all  development  tests  are  properly  time  phased,  that  adequate  resources 
(for  example,  test  articles,  test  facilities,  funding,  and  manpower)  are  available,  and  that 
duplicative  or  redundant  testing  is  eliminated,  a  properly  integrated  test  program  is 
required.  This  activity  must  start  early  in  concept  development  and  continue  into  FSD. 
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AREA  OF  RISK 

The  ultimate  objective  of  a  failure  reporting,  analysis,  and  corrective  action  system 
(FRACAS)  is  to  devise  corrective  actions,  which  prevent  failure  recurrence,  for 
incorporation  into  the  system  or  equipment.  Although  there  are  several  military  standards, 
such  as  Military  Standard  (MIL-STD)  785B  (reference  (e))  and  MIL-STD  781 C  (reference 
(f)),  that  require  FRACASs,  the  implementation  of  these  requirements  has  been  managed 
poorly,  defined  improperly,  and  undisciplined.  The  flow  down  of  requirements  from  prime 
contractor  to  subcontractors  has  not  been  uniform,  analysis  of  all  failures  has  not  been 
required,  the  timely  closeout  of  failure  reports  has  been  overlooked,  and  systems  for 
alerting  higher  management  to  problem  areas  have  been  missing. 


CORPORATE  MANAGEMENT 
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OUTLINE  FOR  REDUCING  RISK 

•  A  central  technical  organization  is  responsible  for  implementation  and  monitoring. 

•  A  FRACAS  is  initiated  at  the  piece  part  level. 
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•  Uniform  requirements  are  imposed  on  subcontractors,  prime  contractors,  and 
Government  activities. 

•  All  failures  are  reported. 

t 

•  All  failures  are  analyzed  to  sufficient  depth  to  identify  failure  cause  and  necessary 
corrective  actions. 

•  All  failure  analysis  reports  are  closed  out  within  30  days  of  failure  occurrence,  or 
rationale  is  provided  for  any  extensions. 

•  Corporate  management  automatically  is  alerted  to  failures  exceeding  closeout 
criteria. 

•  Corporate  management  automatically  is  alerted  to  ineffective  corrective  actions. 

•  Small  subcontractors  lacking  facilities  for  indepth  failure  analysis  arrange  for  the  use 
of  prime  contractor,  Government,  or  independent  laboratory  facilities  to  conduct 
such  analyses. 

•  Criticality  of  failures  is  prioritized  in  accordance  with  their  individual  impact  on 
operational  performance. 


TIMELINE 
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A  FRACAS  will  be  effective  only  if  the  reported  failure  data  is  accurate.  The  failure  reporting 
system  is  initiated  with  the  start  of  the  test  program  and  continues  through  the  early  stages 
of  development. 
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TRANSITION  PLAN 


AREA  OF  RISK 


UNIFORM 

TEST 

REPORT 


Formal  reliability  development  tests  using  the  TAAF  methodology  normally  are  performed 
for  failure  mode  identification  and  elimination.  During  these  tests,  all  results  are  reported 
in  a  format  that  provides  acquisition  managers  with  visibility  of  actual  versus  predicted 
reliability  growth.  Results  from  other  tests  being  performed  during  the  development  and 
transition  phases  usually  are  reported  in  different  formats.  This  change  in  format  precludes 
merger  of  test  results  and  prevents  an  overall  assessment  of  design  maturity  by  acquisition 
managers. 


OUTLINE  FOR  REDUCING  RISK 


•  All  test  results,  including  initial  field  operations,  are  reported  using  the  TAAF  format, 
an  example  of  which  is  shown  in  figure  4-2. 

•  Plotted  results  are  used  to  assess  design  maturity  and  readiness  for  transition  from 
development  to  production. 
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CUMULATIVE  OPERATING  TIME  (HRS) 


Figure  4-2.  Growth  Tracking  Chart 

TIMELINE 


All  test  data  must  be  collected  in  the  specified  TAAF  format  and  analyzed  to  determine 
reliability  growth.  Reporting  test  results  in  the  TAAF  format  begins  with  the  earliest  program 
testing  and  continues  into  service  use  to  allow  a  uniform  baseline  to  evaluate  failures  and 
corrective  actions. 
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TEMPLATE 


There  is  no  way  to  test  all  possible  paths  during  a  development  and  acceptance  test  for  a 
complex  system  involving  immense  logic  complexity.  Some  of  these  paths  eventually  will 
be  exercised  after  the  system  is  deployed  and  some  legitimate  user  interfaces  will  occur 
that  were  not  tested  specifically.  These  will  result  in  software  errors. 


Many  past  studies  on  hardware  illustrate  how  the  cost  of  correcting  a  design  error  multiplies 
if  the  problem  is  not  found  untif  acceptance  testing,  production,  or  deployment.  The  same 
applies  to  software,  but  the  cost  for  correcting  software  design  errors  after  the  design  phase 
multiplies  at  a  much  greater  rate. 


Figure  4-3.  is  based  on  combined  data  from  four  major  contractors  and  shows  a  multiple  of 
100:1  for  cost  to  correct  a  design  error  after  the  system  is  operational. 


Figure  4-3.  Relative  Cost  to  Correct  Design  Error 
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OUTLINE  FOR  REDUCING  RISK 

•  Up  front  money  is  available  for  testing  software  early  in  the  design  phase  to  prevent 
design  and  coding  errors  from  being  discovered  after  deployment. 

•  The  software  design  allows  the  product  to  be  testable.  The  test  group  is  an  active 
participant  in  software  design  reviews  to  ensure  that  the  design  is  testable  to  the 
greatest  degree. 

•  An  independent  test  group  is  used  to  initiate  the  software  test  plan  and  to  initiate 
testing  at  the  functional  module  level  early  in  the  program. 

•  Test  readiness  reviews  are  used  to  ensure  good  software  test  planning. 

•  For  extremely  high  reliability  requirements,  the  verification  and  validation  approach 
is  used.  An  independent  test  group  is  used  to  verify  by  analysis  or  test  every 
important  test  action. 

•  Useful  definitions  of  error  and  failure  are  developed  and  software  reliability  growth  is 
tracked  during  all  test  phases  using  a  closed  loop  failure  reporting  system.  Every 
failure  is  analyzed  placing  special  emphasis  on  resolving  anomalies. 

•  Stress  testing  and  “worst  case”  testing  are  utilized  to  ensure  that  adequate  design 
margins  exist  in  memory  loading,  data  rates,  port  timing,  and  other  critical 
parameters. 

•  Security  requirements  are  considered  during  software  testing. 
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The  best  approach  in  testing  software  is  through  testing  at  each  of  the  early  stages  of 
design  and  coding  to  reduce  the  probability  of  errors  escaping  and  surfacing  during  system 
integration  tests  and  field  use.  Assurance  of  software/hardware  interface  compatibility  is 
obtained  by  exhaustively  testing  the  software  in  a  total  system  test  bed. 
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AREA  OF  RISK 


Design  limit  tests  are  intended  to  ensure  that  system  or  subsystem  designs  are  adequate  to 
meet  specified  performance  characteristics  when  exposed  to  “worst  case”  environmental 
conditions  expected  at  the  extremes  of  the  operating  envelope.  Nevertheless,  test 
environments  often  are  not  representative  of  the  “worst  case”  operating  environment, 
resulting  in  high  risk  of  poor  performance  during  operational  use. 


OUTLINE  FOR  REDUCING  RISK 

•  One  specific  set  of  system-level  test  environments  based  on  expected  operational 
(mission  profile)  environments  is  used. 

•  System-level  operational  test  environments  are  allocated  to  each  subsystem  and 
tailored  to  the  expected  operational  environment  for  each  subsystem. 

•  Design  limit  qualification  test  environments  are  based  on  the  “worst  case”  conditions 
in  the  system  and  subsystem  life  cycle  profiles. 

•  Contractors  are  provided  with  measured  environmental  data  to  use  in  developing 
test  environments. 

•  Test  environments  are  modified  as  additional  environmental  data  become  available. 

•  Failures  occurring  during  design  limit  qualification  testing  are  investigated  thoroughly 
to  determine  the  mechanisms  of  failure,  so  that  corrective  action  can  be  initiated. 
Timeliness  is  important  to  ensure  cost-effective  design  improvements. 
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•  Design  limit  qualification  testing  is  conducted  on  critical  hardware  at  the  lowest  level 
of  assembly. 

•  A  test  history  file  is  maintained  on  design  limit  qualification  tests  for  future  use  on  the 
program  and  as  a  reference  for  new  programs. 

•  Subsystem  qualification  tests  are  scheduled  and  conducted  so  that  completion 
occurs  before  the  production  decision. 
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Design  limit  tests  ensure  that  system  or  subsystem  designs  meet  performance 
requirements  when  exposed  to  environmental  conditions  expected  at  the  extremes  of  the 
operating  envelope — the  “worst  case”  environments  of  the  mission  profile. 
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AREA  OF  RISK 


Life  tests  are  intended  to  assess  the  adequacy  of  a  particular  equipment  design  when 
subjected  to  long-term  exposure  to  certain  mission  profile  environments.  Due  to  the  time- 
consuming  nature  of  these  tests,  various  methods  have  been  used  to  accelerate  test  times 
by  exposure  to  more  stringent  environments  than  those  expected  in  actual  operational  use. 
These  methods  may  give  misleading  results  due  to  a  lack  of  understanding  of  the 
acceleration  factors  involved,  for  example,  recent  attempts  to  develop  accelerated  life  tests 
to  verify  long-term  dormant  storage  requirements  for  missiles. 

Many  weapon  system  programs  are  forced  into  conducting  life  tests  after  the  systems  are 
deployed  and  before  reliability  requirements  are  achieved.  As  a  result,  life  tests  are 
performed  after  the  start  of  production  and  costly  engineering  change  proposals  (ECPs), 
and  retrofit  programs  must  be  initiated  in  an  attempt  to  “get  well”  with  less  than  optimum 
design  solutions. 


OUTLINE  FOR  REDUCING  RISK 

•  Include  life  testing  in  the  overall  system  integrated  test  plan  to  ensure  that  testing  is 
conducted  in  a  cost-effective  manner  and  to  meet  program  schedules. 

•  Use  test  data  from  other  phases  of  the  test  program  to  augment  the  system  and 
subsystem  life  testing  by  reducing  the  time  required  to  prove  that  reliability 
requirements  are  met. 

•  Use  life  test  data  from  similar  equipments  operating  in  the  same  environment  to 
augment  the  equipment  life  testing,  in  order  to  gain  confidence  in  the  design.  For 
example,  this  technique  is  useful  particularly  when  determining  the  long-term 
dormant  life  expectancy  of  a  missile. 
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•  Conduct  early  assessment  of  operational  life  expectancy  through  realistic  life  testing 
that  will  ensure  timely  feedback  of  test  results  to  design  activities. 

•  Develop  realistic  life  test  environments  based  on  operational  mission  profile 
environments.  Experience  gained  from  previous  programs  is  useful  in  developing  life 
test  parameters. 

•  Use  only  proven,  well  understood,  accelerated  testing  techniques  in  the  design  of  life 
tests. 

•  Analyze  failure  data  originating  from  life  tests  in  sufficient  depth  to  identify  the  root 
cause  of  failure,  so  that  the  proper  design  correction  can  be  implemented. 

•  A  well-designed  life  test  is  an  excellent  measure  of  the  level  of  design  maturity. 

•  Fatigue  life  tests  should  be  conducted  to  loading  spectra  that  will  determine  the 
inherent  strength  of  the  parts  so  that  their  lives  can  be  recalculated  should  the 
operational  mission  profile  be  changed  or  revised  test  conditions  differ  from  those 
calculated. 


TIMELINE 
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t  1 

I  Test,  Analyze,  and  Fix  (TAAF) 

f||  ®  raW  Irlpilrll  il 

A  well-designed  life  test  is  an  excellent  measure  of  the  level  of  design  maturity  and  is  used 
to  establish  life  characteristics.  Life  testing  is  integrated  with  other  development  test 
activities  and  is  completed  before  design  release. 
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TEMPLATE 


PflOOUCT 


AREA  OF  RISK 


Many  past  development  contracts  have  not  given  proper  emphasis  to  reliability 
development  testing,  utilizing  the  TAAF  methodology.  Instead,  they  limit  their  approach  to  a 
reliability  test  to  demonstrate  a  numerical  mean  time  between  failure  (MTBF)  requirement. 
This  latter  approach  has  been  ineffective  in  providing  weapon  systems  with  acceptable  field 
reliability.  Reliability  development  testing  (TAAF)  using  simulated  mission  environments 
and  emphasizing  reliability  growth  has  proven  a  more  effective  use  of  limited  test  resources 
and  has  reduced  the  risk  of  allowing  systems  with  poor  reliability  to  transition  from 
development  to  production.  TAAF  is  consistent  with  the  growth  requirement  of  DoD 
Directive  5000.40  (reference  (g)). 


OUTLINE  FOR  REDUCING  RISK 

•  Reliability  development  tests  are  performed  instead  of  reliability  demonstration  tests, 
which  are  nonproductive  cost  and  schedule  drivers. 

•  Reliability  development  test  resources  are  directed  to  subsystems  of  low  (predicted) 
reliability  when  improvement  will  have  a  significant  influence  on  overall  weapon 
system  reliability. 

•  If  subsystems  of  high  (predicted)  reliability  exhibit  reliability  problems  during  other 
development  tests,  such  subsystems  are  incorporated  in  the  reliability  development 
test  program. 

•  For  most  efficient  use  of  test  resources,  reliability  development  tests  are  integrated 
with  other  tests,  such  as  environmental  qualification  tests,  to  avoid  duplication. 
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•  Reliability  development  tests  use  mission  profile  environments. 

•  The  predicted  MTBF  is  at  least  1.25  times  the  required  MTBF  (see  figure  4-4.). 

•  An  initial  MTBF  estimate  of  30  percent  of  the  predicted  MTBF  should  be  used  for 
low  risk  programs.  A  substantially  lower  estimate,  as  low  as  10  percent  in  some 
cases,  should  be  used  for  high  risk  programs. 

•  A  growth  slope  of  0.5  can  be  achieved  by  a  well-executed  program. 

•  There  are  no  random  failures — all  failures  require  analysis  and  implementation  of 
corrective  action  to  prevent  their  recurrence. 

•  Results  of  reliability  development  tests  and  other  development  and  operational  tests 
are  used  to  assess  reliability. 

•  Reliability  development  tests  are  terminated  when  further  tests  produce  insignificant 
improvements. 

•  A  typical  reliability  development  test  example  is  shown  in  figure  4-4.  for  both  low  risk 
and  high  risk  programs. 

EVALUATION  EXPOSURE  HOURS  (LOGARITHMIC) 


100  200  400  600  800  1000  2000  4000  6000  8000  10000 


Figure  4-4.  Reliability  Development  Test  Planning  Model 
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TIMELINE 


TAAF  tests  are  implemented  during  FSD,  to  ensure  the  early  incorporation  of  corrective 
action  necessary  for  continuous  reliability  growth.  TAAF  tests  are  integrated  with  other  test 
activities  and  are  completed  before  the  initial  production  decision. 
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field: 

FEEDBACK* 


Early  feedback  of  problems  occurring  during  initial  use  of  weapon  systems  is  essential  for 
the  elimination  of  unforeseen  design  defects  and  correction  of  problems.  Feedback  of  field 
problems,  however,  is  slow  and  inadequate,  and  failed  parts  are  not  returned  for  analysis 
in  a  timely  manner.  Onsite  engineering  teams  can  provide  adequate  reporting  and  return  of 
parts,  but  the  usual  contractual  approach  to  the  use  of  the  teams  is  to  address 
implementation  at  contractors’  facilities  only  and  not  to  include  provisions  for  service  use  at 
remote  sites. 


OUTLINE  FOR  REDUCING  RISK 

•  Weapon  systems’  contracts  provide  for  an  onsite  engineering  team  to  observe  initial 
operation,  help  in  identifying  problems,  provide  early  feedback  of  field  problems, 
and  provide  sufficient  data  to  allow  design  changes  or  improvements  to  the 
manufacturing  process.  The  duration  of  this  service  is  established  during  contract 
negotiations. 

-  The  types  of  problems  encountered  in  initial  service  operation  of  new  weapon 
systems  require  engineering  solutions. 

-  Solutions  are  enhanced  significantly  by  onsite  engineering  analysis. 

-  Experience  has  demonstrated  the  effectiveness  of  the  onsite  analysis  process 
in  improving  field  reliability  of  weapon  systems. 

—The  final  payoff  of  the  onsite  engineering  team  is  the  improved  reliability  of  the 
system  during  service  operation.  This  is  illustrated  in  figure  4-5.  for  a  recent 
fighter  aircraft  program.  The  reliability  problems  identified  in  service  use 
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contributed  the  major  part  of  the  observed  improvement  in  mean  flight  hours 
between  failure  (MFHBF)  and  reduction  in  discrepancy  reports. 

—  The  onsite  team  is  trained  adequately. 

-  Direct  communication  link  is  maintained  with  the  design  team. 

Onsite  engineering  teams  are  not  used  on  small  programs  where  the  risk  is  low. 
Judgment  is  required  for  effective  use. 


A/C  MFHBF 


DISCREPANCY  gg 
REPORTS 


NUMBER  OF 

FIELD  DISCREPANCY 

REPQRTS/MONTH 


B 
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CALENDAR  TIME  •  MONTHS 

F)gure  4.5.  Typical  Aircraft  Service  Transition  Services 
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Integrated  Taat 
Fatiura  Reporting  System 
Uniterm  Tast  flaport 
Software  tast 
Oasiga  Limit 
Lite 

Tast,  Analyza,  and  Rx  (TMF) 


Early  feedback  of  problems  occurring  during  initial  use  of  weapon  systems  is  essential  for 
elimination  of  unforeseen  design  defects  and  correction  of  problems  caused  by  the 
transition  to  full  rate  production  and  tooling.  Onsite  engineering  teams  are  used  as  soon  as 
field  operations  begin  and  continue  through  service  use  to  improve  the  accuracy,  quantity, 
and  speed  of  reporting  of  field  failures  and  corrective  actions. 
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CHAPTER  5 

INTRODUCTION  FOR  PRODUCTION  CRITICAL  PATH  TEMPLATES 


Solving  the  manufacturing  portion  of  the  equation  is  a  major  factor  in  reducing  the  risk  of 
transition  from  development  to  production.  The  history  of  military  procurements  chronicle 
again  and  again  the  scenario  of  proven  functional  designs  being  introduced  into  the 
manufacturing  process,  only  to  complete  that  process  as  end  products  that  cannot  support 
their  mission  requirements. 

The  DSB  task  force  investigated  transition  matters  related  to  preparation  for  and 
management  of  the  manufacturing  process.  More  specifically,  it  dealt  with  issues  in  such 
areas  as  part  quality  and  management;  the  cause  and  relation  of  workmanship  defects;  the 
vendor  impact  on  quality,  cost,  and  schedule;  the  recipes  for  successful  transition  to 
production;  and  the  associated  transition  management  techniques.  The  task  force  agreed 
that  within  industry  today  there  exists  the  experience,  wisdom,  tools,  and  techniques  to 
successfully  manage  the  transition  process.  However,  based  on  past  transition  experience, 
the  issues  outlined  in  this  section  represent  those  that  have  been  especially  troublesome 
and  require  special  initiatives  and  discipline  to  manage  effectively.  Consequently,  the 
implementation  of  the  concepts,  techniques,  and  procedures  specified  in  this  section  will 
reduce  significantly  the  risk  of  transition  from  development  to  production. 
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AREA  OF  RISK 


Involvement  of  production  and  manufacturing  engineering  only  after  the  design  process 
has  been  completed  is  a  fundamental  error  and  a  major  transition  risk.  Consequences  of 
late  involvement  are  (I)  an  extended  development  effort  required  for  redesign  and  retest  of 
the  end  item  for  compatibility  with  the  processes  and  procedures  necessary  to  produce  the 
item,  and  (2)  lower  and  inefficient  rates  of  production  due  to  excessive  changes  in  the 
product  configuration  introduced  on  the  factory  floor.  Increased  acquisition  costs  and 
schedule  delays  are  the  result  of  this  approach. 


OUTLINE  FOR  REDUCING  RISK 

•  Documented  early  planning  that  focuses  on  the  specifics  of  the  fabrication  practices 
and  processes  required  to  build  the  end  item  is  initiated  while  the  design  is  fluid  and 
completed  before  the  start  of  rate  production.  Documenting  this  process  constitutes 
a  manufacturing  plan. 

•  The  following  represent  the  key  elements  of  a  manufacturing  plan: 

—  Master  delivery  schedule  that  identifies  by  each  major  subassembly  the  time 
spans,  need  dates,  and  who  is  responsible. 

—  Durable  tooling  requirements  to  meet  increased  production  rates  as  the 
program  progresses. 

—  Special  tools. 

—  Special  test  equipment. 

—Assembly  flowcharts. 
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—Test  flowcharts. 

—  Receiving  inspection  requirements  and  yield  thresholds. 

—  Production  yield  thresholds. 

—  Producibility  studies. 

—  Critical  processes. 

—  Cost  and  schedule  reports. 

—  Trend  reports. 

—  Inspection  requirements. 

—  Quality  plan. 

—  Fabrication  plans. 

—  Design  release  plan. 

—  Surge  and  mobilization  planning. 

—  Critical  and  strategic  materials. 

—  Labor  relations. 

—  Manpower  loading. 

—  Training. 

—  Training  facility  loading. 

—  Production  facility  loading  and  capacity. 

—  Machine  loading. 

—  Capital  investment  planning. 

—  Make  or  buy  criteria. 

—Subcontractor  and  vendor  delivery  schedules. 

-Government-furnished  material  demand  dates. 

—Work  measurement  planning. 

—  Energy  management  audits. 

•  The  following  elements  also  may  be  considered  when  generating  a  manufacturing 
plan.  They  usually  are  influenced  by  unique  aspects  of  the  acquisition,  capabilities  of 
the  contractor,  or  initiatives  of  the  military  procurement  agency. 

—  Project  and  functional  personnel  in  manufacturing  are  collocated. 

—  Engineering  and  manufacturing  test  equipment  are  built  alike. 

—  Assembly  planning  is  verified  before  rate  production. 

—  Specify  that  a  part  of  design  engineers’  time  be  spent  on  the  factory  floor. 

—  Assembly,  inspection,  test,  and  rework  are  combined  in  unit  work  cells,  when 
appropriate. 

—  Development  hardware  is  inspected  by  production  line  inspectors. 

—  Production  personnel  participate  in  building  development  hardware. 
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•  The  overall  manufacturing  strategy  developed  earlier  in  the  acquisition  cycle  is 
implemented  by  production  planning  activities. 

•  The  manufacturing  plan  is  verified  and  progress  against  the  plan  is  monitored  by  a 
series  of  contractual  and  internal  production  readiness  reviews. 

—  Reviews  include  both  prime  contractor  and  subcontractor.  It  is  the  prime 
contractor’s  responsibility  to  ensure  that  production  readiness  reviews  are 
conducted  at  the  subcontractor’s  facility. 

— These  reviews  are  staffed  with  knowledgeable  personnel  (that  is,  a  mixture  of 
manufacturing  and  design  engineering  people  from  outside  the  line  organization 
doing  the  work). 

—  The  depth  of  these  reviews  is  similar  to  that  of  the  design  reviews  with 
participation  by  a  similar  level  of  qualified  people  in  the  areas  of  design  and 
manufacturing  engineering. 

TIMELINE 


The  manufacturing  plan  identifies  the  approach  for  effective  fabrication  of  the  product 
design.  Manufacturing  planning  activities,  concurrent  with  development  activities,  are 
essential. 
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TEMPLATE 
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AREA  OF  RISK  “ 

The  introduction  of  a  recently  developed  item  to  the  production  line  brings  new  processes 
and  procedures  to  the  factory  floor.  Changes  in  hardware  or  workflow  through  the 
manufacturing  facility  increase  the  possibility  of  work  stoppage  during  rate  production. 
Failure  to  qualify  the  manufacturing  process  before  rate  production  with  the  same  emphasis 
as  design  qualification — to  confirm  the  adequacy  of  the  production  planning,  tool  design, 
manufacturing  process,  and  procedures — can  result  in  increased  unit  costs,  schedule 
slippage,  and  degraded  product  performance. 


OUTLINE  FOR  REDUCING  RISK 

•  The  work  breakdown  structure,  production  statement  of  work  (as  identified  in  the 
contract),  and  transition  and  production  plans  do  not  contain  any  conflicting 
approaches.  Any  discrepancies  among  these  documents  are  identified  and  resolved 
before  production  is  started. 

•  A  single  shift,  8-hour  day,  5-day  workweek  operation  is  planned  for  all  production 
schedules  during  initial  startup.  Subsequent  manpower  scheduling  is  adjusted  to 
manufacturing  capability  and  capacity  consistent  with  rate  production  agreements. 

•  The  drawing  release  system  is  controlled  and  disciplined. 

—  Manufacturing  has  the  necessary  released  drawings  to  start  production. 

—  No  surge  in  engineering  change  proposal  (ECP)  traffic  from  design  or 
producibility  changes  occurs. 
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— “Block  changes”  to  the  production  configuration  are  minimized.  (A  consistent 
configuration  that  does  not  need  any  block  changes  is  an  indication  of  low  risk.) 

•  The  manufacturing  flow  minimizes  tooling  changes  and  machine  adjustments  and 
ensures  that  alternate  flow  plans  have  been  developed. 

•  A  mechanism  is  established  that  ensures  the  delivery  of  critical,  long  lead  time  items 
4  to  6  weeks  before  required. 

•  All  new  equipment  or  processes  that  will  be  used  to  produce  the  item  are  identified. 

— Qualified/trained  personnel  are  assigned  to  operate  the  new  equipment  and 
processes. 

— “Hands  on”  training  is  accomplished  with  representative  equipment  and  work 
instructions.  (See  Productivity  Center  template.) 

•  Hardware  and  other  resources  are  allocated  to  “proof  of  design”  models  for  data 
package  validation,  and  to  “proof  of  manufacturing”  models  for  implementation 
prove-out  and  production  equipment  troubleshooting.  Quantities  of  the  “proof  of” 
models  are  decided  jointly  by  the  customer  and  contractor  depending  on  the  nature 
and  complexity  of  the  program. 

•  The  manufacturing  process  is  qualified  both  at  prime  contractors  and  all  major 
subcontractors. 

TIMELINE 
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The  manufacturing  process  required  to  produce  an  item  significantly  influences  the  design 
approach  and  product  configuration.  Therefore,  the  manufacturing  process  is  qualified  with 
enough  time  for  design  or  configuration  changes  to  be  introduced  in  the  baseline  product 
configuration  before  low  rate  production  commences. 
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TEMPLATE 


AREA  OF  RISK 
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Most  military  programs  require  MIL-STD  parts  in  weapon  and  support  systems.  This 
practice  has  left  much  to  be  desired  in  its  ability  to  ensure  delivery  of  high  quality,  reliable 
parts  to  contractors.  In  self-protection,  users  must  conduct  intensive  screening  and 
inspection  at  their  own  facilities,  to  provide  an  acceptable  product  to  the  production  line. 
Semiconductors  in  particular  have  played  a  major  role  in  increasing  the  cost  and  risk  of 
producing  a  reliable  product,  in  some  cases  showing  defect  rates  of  3  to  12  percent  during 
user  rescreening. 


OUTLINE  FOR  REDUCING  RISK 

•  Receiving  inspection  is  more  effective  than  source  inspection: 

—Suppliers  tend  to  ship  better  quality  products  to  customers  performing  receiving 
inspection  rather  than  source  inspection. 

— Receiving  inspection  costs  typically  are  less  than  source  inspection. 

—Typically,  more  lots  per  man-hour  can  be  inspected  at  receiving  than  at  source 
inspection. 

•  One  hundred  percent  rescreening  of  semiconductors  reduces  risk  and  usually  is 
cost-effective.  Departures  from  100  percent  rescreening  are  appropriate,  provided 
they  are  supported  by  sound  technical  and  cost  rationale.  Factors  influencing  a 
departure  might  include  the  use  of  mature  technology  parts,  demonstrated  ability  of 
the  supplier  to  deliver  consistently  quality  products,  and  test  and  failure  cost  data. 
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The  following  represents  a  minimal  baseline  program  to  be  conducted  at  the  user's 
facility: 

-  Perform  particle  induced  noise  (PIN)  testing,  at  a  minimum,  on  ail  hybrids  and 
preferably  on  all  semiconductors  with  cavities  when  used  in  critical  applications. 

—  Perform  electrical  test  at  -  55°C,  +  25°C,  and  + 1 25°C. 


Typical  costs  (1982  dollars)  for  the  above  tests: 


—  Transistor/transistor  logic  (TTL)  integrated  circuits  S  .68 

—  Complimentary  metal  oxide  semiconductor  (CMOS)  logic 

integrated  circuits  .81 

—  Linear  integrated  circuits  1 .04 

—  Memories/microprocessors  1 .45 

—  Transistors/diodes  .74 


Typical  costs  (1982  dollars)  for  parts  replacement  if  the  defect  is  found  at  a  higher 
level  of  assembly: 


—  Printed  wiring  assembly  $  50 

—  Line  replaceable  unit  500 

—  System  1,500 

-Field  15,000 


•  Performing  destructive  physical  analysis  (DPA)  at  the  user’s  facility  also  can  detect 
faulty  parts,  can  verify  suppliers’  processes,  and  is  a  good  adjunct  to  the  rescreening 
program. 

•  Small  users  can  use  an  independent  test  laboratory  to  conduct  rescreening  if  they 
lack  the  necessary  test  equipment.  Costs  to  conduct  this  screening  are  similar  to 
those  quoted  above. 

•  Receiving  inspection  and  rescreening  exert  contractual  leverage  on  part  suppliers  to 
improve  overall  quality  of  the  product  and  ultimately  to  reduce  the  cost  of  parts  to  the 
user. 

•  Pretin  component  leads  and  conduct  a  solderability  test  at  incoming  inspection. 

•  Piece  part  control  includes  provisions  for  screening  of  parts  (especially  mechanical 
and  electrical  components,  as  well  as  electronic  devices),  to  ensure  proper 
identification  and  use  of  standard  items  already  in  the  Military  Service  logistics 
system. 
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TIMELINE 


A  kev  element  of  parts  control  is  an  established  policy  that  ensures  that  certain  steps  are 
taken  early  in  the  buildup  of  the  first  hardware  items  to  control  part  quality  (both  electrical 

and  mechanical). 


i 
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TEMPLATE 
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AREA  OF  RISK 


Over  the  years,  the  percentage  of  major  weapon  systems  that  are  subcontracted  has 
grown,  reaching  as  much  as  80  percent  in  some  cases.  Hence,  reliance  on  subcontractors 
and  upon  the  skills  of  prime  contractors  to  manage  their  subcontractors  and  suppliers  has 
increased.  An  informal  poll  of  ten  prime  contractors  averaging  about  ten  major  programs 
each  resulted  in  statements  that  nearly  half  their  programs  were  in  schedule  or  cost  trouble 
because  of  major  subcontractor  problems.  Clearly,  the  effective  management  of 
subcontractors  needs  more  emphasis  within  industry  and  in  the  Government’s 
management  of  prime  contractors  if  there  is  to  be  a  smooth  transition  to  production. 


OUTLINE  FOR  REDUCING  RISK 

•  Request  for  proposals  (RFPs)  for  prime  contractors  require  responses  from  bidders 
with  equitable  emphasis  on  subcontractor  management  planning  versus  in-house 
management.  Responses  include  the  following: 

—  Prime  contractor’s  organization  for  managing  subcontractors. 

—  Plans  for  onsite  evaluation  of  potential  subcontractors  before  source  selection. 

-Tasks  and  associated  payment  plans  to  ensure  that  required  up-front 
subcontractor  activities  are  visible. 

—  Plans  for  program  reviews,  vendor  audits,  and  production  readiness  reviews. 

•  Military  program  managers  and  prime  contractors  conduct  vendor  conferences  that 
address  the  following: 
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—  Educate  each  subcontractor  thoroughly  on  the  requirements  in  his  or  her 
contract,  as  well  as  the  key  elements  of  the  prime  contract. 

—  Communicate  to  the  subcontractors  what  is  required  of  them. 

—  Provide  an  awareness  of  their  role  in  the  total  weapon  system  acquisition. 

—  Allocate  resources  to  do  the  job  right. 

—  Recognize  and  (when  appropriate)  reward  good  performance. 

•  Prime  contractors  establish  resident  interface  at  critical  subcontractors  before 
production  start. 

•  Prime  contractors  maintain  a  roster  of  “subcontractor  assist”  personnel  for  surprise 
problems. 

•  Budget  for  both  resident  and  “subcontractor  assist”  teams  to  be  available  on 
demand  with  well-qualified  technical,  process,  manufacturing,  and  procurement 
people. 

•  Proper  funding  is  committed  to  conduct  the  above  guidelines  during  the  early  design 
phases,  to  ensure  adequate  support  to  procurement.  An  estimate  for  an  80  percent 
subcontracted  program  amounts  to  3  to  4  percent  of  full-scale  engineering 
development  costs. 


TIMELINE 


Informal  and  formal  program  reviews  are  an  essential  ingredient  of  effective  subcontractor 
control  during  the  development  process.  The  prime  contractor  shall,  on  a  regular  basis, 
evaluate  the  “real”  progress  made  by  the  subcontractor  through  such  reviews. 
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AREA  OF  RISK 


High  defect  rates  in  a  manufacturing  process  drive  up  production  costs  because  of  higher 
rework  and  scrap  costs.  Product  quality  is  a  function  of  the  variability  of  defects,  that  is,  the 
higher  the  number  of  defect  types,  the  lower  the  quality  and  vice  versa.  Lack  of  an  effective 
defect  information  and  tracking  system  not  only  increases  production  costs  but  also 
degrades  the  product's  performance  in  the  field. 


OUTLINE  FOR  REDUCING  RISK 

•  Types  of  assembly  defects  are  identified  in  terms  of  specific  data  categories  and 
priorities  for  corrective  action.  (See  figure  5-1.,  which  applies  to  electronic  parts. 
Similar  figures  are  derived  for  other  categories  of  component  parts.) 


DEFECT  DATA  CATEGORIES 


Figure  5-1.  Assembly  Defects 
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Effectiveness  of  a  time-phased  corrective  action  program  is  tracked  (see  figure  5-2.' 


INSPECTION 
/  COMPARATOR 

/  INTEGRATED  COMPUTER  GRAPHICS  -  COLORED 
^  COLOR-CODED  PART  BINS 

CAM-AIOEO  DEFECT  IDENTIFICATION 
BACK  LIGHT  PROJECTION 
COMMON  BUILD.  SOLDER  AND  TEST  FIXTURE 

\_  LEAOFORMED  LEADWIREO 

IN-LINE  SEQUENCE  AND  INSERTION  CHECK 
AUTOMATIC  SEQUENCE  KITS 
COMPUTER-AIDED  ASSEMBLY 
V AUTOMATED  ASSEMBLY  AID 


S  O  N  0  J  M  A  «  j  J  A  S  O  N  0 

■  I960  'I . 19C1  f 


Figure  5-2.  Corrective  Action  Program 

•  Inspection  and  test  yields  and  hardware  throughputs  are  monitored  continuously 
with  predetermined  action  thresholds  (see  figure  5-3.) 

10  UNIT 

MOVING  AVERAGE 


Q  5  10  15  20  25  30  35  40  45  50  55  60 

UNITS  PASSED 

Figure  5-3.  Performance  Threshold  Tracking 


—  Caution  threshold  requires  engineering  action: 
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•  Seventy-two-hour  maximum  response  time. 

•  Daily  reporting  to  program  management  until  caution  thresholds  are 
exceeded. 

—  Alert  threshold  requires  functional-level  management  action: 

•  Seventy-two-hour  maximum  response  time. 

•  Daily  progress  reports  to  program  management  until  all  thresholds  are 
exceeded. 

—  Alarm  threshold  requires  full-time  team  action: 

•  Program  manager  constitutes  team  within  24  hours. 

•  Action  is  implemented  and  reported  to  program  management  within  72 
hours. 

•  Daily  reports  to  program  management  until  thresholds  are  exceeded. 

•  A  feedback  system  to  factory  personnel  and  manufacturing  supervisors  is 
established. 

•  Factory  policy  adequately  reflects  the  criticality  of  its  defect  information  and  tracking 
system. 

•  Critical  process  yields  are  monitored  and  tracked  to  ensure  consistency  of 
performance  (see  figure  5-4.) 


JfMAMJJASONO  1932 

totals 


Figure  5-4.  Production  “Rate  Test”  Defects 
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TIMELINE 
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— 

A  management  commitment  to  defect  “prevention”  is  the  prime  ingredient  of  a  sound 
defect  control  program.  A  management  policy  on  defect  control  is  established  during  the 
development  phase.  This  policy  will  require  management  involvement  in  the  review  of 
defect  analyses  and  an  emphasis  on  defect  “prevention”  that  is  flowed  down  to  all 
subcontractors. 
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TEMPLATE 
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AREA  OF  RISK 


Tools  are  auxiliary  devices  and  aids  used  to  assist  in  the  manufacturing  and  test  processes. 
They  range  from  special  handling  devices  to  ensure  personnel  and  equipment  safety,  to 
equipment  required  for  methods  planning  to  achieve  the  designed  quality,  rate,  and  cost. 
The  risks  associated  with  improper  tool  planning  and  proofing  affect  cost,  quality,  and 
ability  to  meet  schedules.  Improper  tools  prevent  workers  from  achieving  desired 
production  rates,  fail  to  prevent  or  perhaps  even  contribute  to  errors  in  the  build  process, 
and  cause  more  man-hours  of  labor  to  be  expended  in  accomplishing  a  task  than  were 
planned. 

OUTLINE  FOR  REDUCING  RISK 

•  A  tooling  philosophy  is  documented  as  a  part  of  the  early  manufacturing  planning 
process  and  concurrent  with  production  design. 

•  A  detailed  tooling  plan  is  developed  that  defines  the  types  “hard”  or  “soft,”  and 
quantities  required  for  each  manufacturing  step  and  process. 

•  A  requirement  is  included  for  a  similar  plan  for  each  subcontractor  and  its 
implementation  is  disciplined. 

•  Each  tool  is  proofed  rigorously  before  its  initiation  into  the  manufacturing  process  to 
verify  performance  and  compatibility  with  its  specification. 

•  Strict  tool  configuration  management  is  maintained. 
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•  An  effective  tooling  inventory  control  system  is  established  and  maintained  to 
facilitate  continuous  accountability  and  location  control. 

•  A  routine  maintenance  and  calibration  program  is  established  and  conducted  to 
maintain  tool  serviceability. 

•  Manufacturing  engineering  and  tool  designers  are  collocated  with  design  engineers 
when  practical,  and  CAD/CAM  systems  are  used  in  tool  design  and  fabrication. 


TIMELINE 
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Tool  planning  encompasses  those  activities  associated  with  establishing  a  detailed 
comprehensive  plan  for  the  design,  development,  implementation,  and  certification  of 
program  tooling.  Tool  planning  and  design  activities  start  early  in  the  development  phase. 
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AREA  OF  RISK 


Special  Test  Equipment  (STE)  is  a  key  element  of  the  manufacturing  process.  It  is  STE  that 
tests  an  article  (or  final  product)  for  performance  after  it  has  completed  in-process  tests  and 
inspections,  final  assembly,  and  final  visual  inspection.  Late  STE  design  activities  and  the 
lack  of  the  availability  of  qualified  STE  on  the  factory  floor  create  unique  technical  risks. 
These  risks  include  inconsistent  final  test  measurements  (when  compared  to  test 
procedures  used  during  the  successful  development  program),  false  alarm  rates  that  result 
in  needless  troubleshooting  and  rework  of  production  hardware,  and  poor  tolerance 
tunneling  that  causes  either  rejection  of  good  hardware  or  the  acceptance  of  hardware  with 
inadequate  performance.  Program  consequences  in  this  situation  are  schedule  delays, 
increased  unit  costs,  and  poor  field  performance  of  delivered  hardware. 


OUTLINE  FOR  RISK  REDUCTION 

•  A  thorough  factory  test  plan  is  developed  before  detailed  design  of  prime  equipment. 

•  Adequate  prime  equipment  designer  input  and  concurrence  on  test  requirements 
and  test  approach  is  required. 

•  Test  equipment  engineers  and  maintainability  engineers  participate  in  prime 
equipment  design  and  partitioning,  test  point  selection,  built-in  test  design,  and 
design  for  test  and  maintenance  as  well  as  function. 

•  Prime  and  STE  systems  design  personnel  are  collocated  when  practical. 

•  The  test  approach  for  completeness  of  test  is  analyzed,  and  a  feedback  loop  to 
correct  test  escapes  is  provided. 
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•  Test  tolerance  strategy  is  employed  to  catch  problems  at  the  lowest  level,  but  does 
not  cause  excessive  rejection  of  an  adequate  product.  Tolerance  incompatibility  with 
higher-level  test  is  corrected. 

•  The  capabilities  of  the  prime  equipment  are  understood  and  utilized  fully  to  achieve 
simplifications  in  STE. 

•  Design  strategies  are  used  in  test  equipment  that  simplify  tolerance  changes  and 
enable  tests  to  be  readily  added  and  deleted.  “Go/no  go”  tests  are  minimized. 

•  Manual  intervention  capability  is  provided  in  automated  test  equipment  so  that  the 
equipment  can  be  used  while  final  software  debugging  is  in  process  (this  also  can 
aid  in  debugging). 

•  Brassboard  prime  equipment  is  used,  when  appropriate,  to  begin  debugging  test 
equipment  (this  can  enhance  test  equipment  schedules). 

•  Prime  equipment  design  personnel  are  assigned  as  part  of  the  test  equipment 
integration  and  verification  effort. 

•  Adequate  time  is  allotted  for  test  equipment  software  debugging  and  compatibility 
verification. 

•  Government  certification  of  factory  test  equipment  is  required,  as  well  as  re¬ 
certification  if  significant  product  and  test  equipment  changes  occur. 

•  A  thorough  and  realistic  rate  analysis  is  performed  to  avoid  shortages  of  test 
equipment  (or  overbuying).  Considered  in  this  analysis  are  the  number  of  expected 
failures  in  prime  and  test  equipment  in  various  phases  of  the  program,  and 
equipment  requirements  to  support  qualification  test,  TAAF,  engineering  problem¬ 
solving,  and  overhaul  and  repair. 

•  Automated  test  techniques  are  used  when  rate  requirements  on  the  program 
warrant  the  investment. 
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STE  should  be  designed,  qualified,  and  used  as  early  as  possible  to  ensure  a  uniform  final 
product  test  from  development  through  production  transition.  The  STE  design  should 
commence  during  the  late  phases  of  advanced  development  (that  is,  before  Milestone  II) 
and  STE  should  be  qualified  before  rate  production. 


5-22 


DoD  4245.7-M 


This  Page  Intentionally  Left  Blank 


5-23 


DoD  4245. 7-M  TEMPLATE 


[CQMPUTEFr-AIDEQ 
MFCS  (CAM)*- 


AREA  OF  RISK 

The  transition  of  a  qualified  design  to  the  manufacturing  process  historically  has  been 
accomplished  via  a  "drawing  package,”  including  not  only  drawings  but  also  a  large 
number  of  related  documents,  truly  a  massive  amount  of  paperwork.  Generation  of  this 
paper  lengthens  the  period  of  transition,  impedes  rapid  and  accurate  communication 
between  the  design  and  manufacturing  functions  during  this  highly  volatile  period,  and 
introduces  numerous  errors  via  the  drawing  package.  Even  some  facilities  that  have 
invested  heavily  in  CAD  continue  to  transfer  their  designs  to  the  factory  on  paper.  Once  the 
drawing  package  is  available,  many  production  facilities  continue  to  utilize  outdated  high 
risk  manual  operations  both  to  duplicate  the  design  (“build  to  print")  in  rate  production  and 
to  manage  the  manufacturing  process. 


OUTLINE  FOR  REDUCING  RISK 

•  The  development  of  software  tools  for  common  use  by  industry  is  supported  by  the 
Department  of  Defense  with  appropriate  resources  and  coordination  efforts. 

•  A  common  data  base  between  the  design  and  manufacturing  functions  has  inherent 
technical  problems  but  has  the  highest  potential  payoff  in  product  quality  and 
productivity. 

•  Implementing  automated  manufacturing  and  control  functions  can  reduce  transition 
time  by  50  percent. 

•  Using  computers  to  control  manufacturing  operations  (fabrication,  assembly,  test, 
and  inspection)  and  to  collect  shop  floor  data  can  increase  productivity,  can  reduce 
required  shop  floor  space,  and  can  improve  product  quality. 
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•  Use  of  computers  to  control  material  flow  and  maintain  inventory  and  in-process 
data  significantly  reduces  inventory  investments  and  storage  space. 

•  Tooling  redesign  occurs  when  product  design  changes.  Using  CAD  reduces  these 
design  iterations.  Therefore,  using  CAD  for  the  product  design  and  the  additional 
use  of  CAD  for  tool  design  can  reduce  tooling  costs  by  50  percent. 

•  Top-down  strategy  for  implementing  CAM  usually  increases  return  on  investment 
(as  opposed  to  replacing  in-kind  capability,  or  bottom-up). 

•  Training  and  retraining  plans  to  maintain  employee  morale  and  productivity  are 
included  in  a  company’s  strategy. 

•  See  template  on  CAD. 


TIMELINE 


Contractors  using  CAM  integrated  with  CAD  are  experiencing  improved  productivity.  With 
manufacturing  personnel  involved  in  the  design  process,  a  common  CAD/CAM  data  base 
can  be  established  resulting  in  reduced  risk  in  the  transition  from  development  to 
production. 
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Environmental  stress  screening  (ESS)  is  a  manufacturing  process  for  stimulating  parts  and 
workmanship  defects  in  electronic  assemblies  and  units.  Although  ESS  has  been  proven  to 
reduce  field  failure  rates  by  20  to  90  percent  (reducing  life  cycle  costs)  and  to  reduce  in- 
plant  failure  rates  by  as  much  as  75  percent  (reducing  production  costs),  its  use  is  still  not 
accepted  universally  by  many  contractors  as  a  standard  part  of  their  manufacturing 
process.  When  ESS  also  is  performed  during  development,  it  helps  to  ensure  that  the 
electronics  hardware  performs  on  demand,  that  the  most  effective  screening  levels  are 
determined  before  high  rate  production,  and  that  possible  part  type  and  vendor  problems 
are  discovered  early.  Analysis  of  failures  experienced  on  unscreened  developmental 
systems  has  indicated  that  60  percent  are  due  to  workmanship,  30  percent  are  due  to  bad 
parts,  and  only  10  percent  are  design  problems.  ESS  should  not  be  confused  with 
environmental  qualification  testing  (which  is  designed  to  demonstrate  design  maturity). 

OUTLINE  FOR  REDUCING  RISK 

•  ESS  procedures  are  established  during  development. 

•  Temperature  cycling  and  random  vibration  are  effective  environmental  stress 
screens  and  are  performed  on  1 00  percent  of  electronic  products  (it  is  not  done  on  a 
sampling  basis). 
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•  The  predominant  factors  in  temperature  cycling  are: 

—  Rate  of  change  of  temperature. 

—  Minimum  and  maximum  range  of  temperature. 

—  Number  of  cycles. 

—  Level  of  assembly  on  which  performed. 

•  The  predominant  factors  in  random  vibration  are: 

— Spectral  density. 

— Lower  and  upper  frequency  limits. 

— Axis  of  stimulation. 

— Level  of  assembly. 

— Duration  of  screen. 

•  Random  vibration  stimulates  more  defects  than  fixed  or  swept  sine  vibration  of 
similar  levels  of  excitation. 

•  There  are  many  technical  and  cost  benefit  tradeoffs  to  be  made  in  designing  an  ESS 
program.  A  particularly  useful  document  in  making  tradeoff  decisions  is  the 
Environmental  Stress  Screening  Guidelines  for  Assemblies.1  A  screening  guidelines 
document  for  parts  will  be  published  by  the  IES  in  late  1985. 

•  Recommended  starting  conditions  are: 

— Random  Vibration: 

•  Spectral  density:  6g  rms 

•  Frequency  limits:  lOO-IOOOHz 

•  Axis:  3 

•  Duration:  10  min. 

— Temperature  Cycling: 

•  Rate:  10°C/minute 

•  Range:  -40°C  to  60°C 

•  Number  of  cycles:  15  (last  must  be  failure  free) 

•  Power:  On  (except  cool  down) 

•  For  greatest  return  on  investment,  vigorous  corrective  actions  are  made  to  adjust 
manufacturing  process  to  minimize  recurrence  of  defects. 

•  The  ESS  program  is  a  dynamic  one.  Procedures  are  adjusted,  as  indicated  by 
screening  results,  to  maximize  finding  defects  efficiently. 


’Sponsored  by  the  Institute  of  Environmental  Sciences  (IES),  September  1984. 


5-27 


DoD  4245. 7- M 


•  Objective  of  ESS  is  not  to  find  design  defects,  although  such  may  be  a  by-product. 


•  Appropriate  screening  for  manufacturing  defects,  as  an  acceptance  test,  is 
developed  for  other  than  electrical  and  electronic  products. 

TIMELINE 


Production 

Manufacturing  Plan 
Qualify  Mfg.  Procasi 
Pi  tea  Part  Control 
Subcontractor  Control 
Oafoct  Control 
Tool  Planning 

Special  Tatt  Equipment  { STE ) 
Computar-Aidad  Mfg.  (CAM) 


Manufacturing  Screening 


ESS  techniques  precipitate  assembly  and  workmanship  defects,  such  as  poor  soldering  or 
weak  wire  bonds  during  the  assembly  process. 
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CHAPTER  6 

INTRODUCTION  FOR  TRANSITION  PLAN  CRITICAL  PATH  TEMPLATE 


The  fundamental  purpose  of  the  transition  plan  is  to  provide  the  integration  methodology 
that  will  tie  together  the  application  of  templates  within  the  context  of  the  industrial  process. 
To  this  end,  it  should  be  viewed  not  as  a  management  procedure  but  as  a  technical 
evaluation  tool. 

This  evaluation  process  begins  first  by  comprehending  fully  the  technical  requirements  of 
the  product  and,  with  that  understanding,  preparing  a  contractor  transition  plan 
(Government-required  and-funded)  at  the  start  of  engineering  development.  The  outlines 
for  reducing  risk,  contained  in  the  preceding  templates,  form  the  basis  upon  which  the 
transition  plan  is  developed  along  with  the  means  by  which  design  readiness  and  maturity, 
test  readiness  and  maturity,  and  manufacturing  readiness  and  maturity  are  assessed 
continuously  for  the  build-up  of  risk. 

An  additional  ingredient  of  the  transition  plan  is  provision  of  the  means  and  explanation  of 
the  procedures  that  clearly  delineate  the  timing  of  the  engineering  disciplines,  criteria  that 
are  to  be  satisfied  while  carrying  out  each  discipline,  data  required  to  assess  the  criteria, 
and  the  significant  risk-driving  relationships  between  the  templates  contained  in  this 
document. 

The  final  objective  of  the  transition  plan  is  to  provide  visibility  on  how  well  the  template 
generated  actions  for  reducing  risk  are  being  executed.  Therefore,  progress  reports  should 
be  compared  regularly  against  the  transition  plan. 
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TEMPLATE 


>  TRANSITION!  PEANUT 


AREA  OF  RISK 


In  the  past,  a  lack  of  formal  transition  planning  has  contributed  significantly  to  the  problems 
encountered  in  the  transition  from  development  to  production.  One  of  the  major  causes  has 
been  a  Government/industry  attitude  that  the  performance  parameters  must  be  achieved 
during  engineering  development  before  expending  funds  to  achieve  production  objectives. 
While  there  were  a  number  of  milestone-oriented  Government  requirements  during  the 
development  phase  and  before  the.  start  of  production,  these  were  really  stand-alone 
requirements  generally  used  to  verify  the  design’s  performance  goals  or  as  negotiation 
materials  not  having  a  smooth  transition  as  an  end  objective. 


OUTLINE  FOR  REDUCING  RISK 


•  Formal  Government  policies'  and  specified  contractual  requirements  that  lay  the 
groundwork  for  planning,  programing,  and  executing  specific  actions  during  the 
development  phase  to  ensure  a  smooth  and  successful  transition  to  production  are 
set  forth  in  DoD  Directive  4245.6  (reference  (h))  and  DoD  Directive  4245.7 
(reference  (i)). 

•  The  Government  program  manager  is  required  to  fund  and  execute  a  contractor- 
developed  transition  plan,  initially  prepared  no  later  than  the  start  of  engineering 
development  and  continually  updated  until  rate  production  is  achieved. 
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•  A  sample  transition  plan  outline  includes,  but  is  not  limited  to,  consideration  of  all 
templates  in  this  Manual.  The  transition  plan  integrates  the  design,  test,  and 
manufacturing  activities  in  order  to  reduce  data  requirements,  duplication  of  effort, 
costs,  and  schedule.  It  identifies,  for  example,  test  and  manufacturing  issues  that 
impact  design,  and  design  issues  that  affect  test  and  manufacturing.  The  transition 
plan  is  a  major  means  of  implementing  the  manufacturing  strategy  described  in  one 
of  the  management  templates. 

•  Development  contracts  contain  the  requirement  for  a  formal  design-to-unit 
production  cost  program  and  provisions  for  proof  of  manufacturing  methods  and 
processes.  Funding  is  provided  to  the  contractors  for  these  areas  of  activity. 

•  The  contractor’s  approach  to  obtaining  both  producibiiity  in  the  design  and  an 
effective  transition  from  development  to  production  is  solicited  in  the  RFP  and 
weighted  heavily  in  source  selection. 

•  Formal  production  readiness  reviews  (PRRs)  are  conducted  jointly  by  the  customer 
and  the  contractor  during  the  development  effort  and  completed  before  the 
production  decision.  Participants  in  these  reviews  are  qualified  and  experienced 
both  in  technical  aspects  of  the  product  and  the  manufacturing  processes  proposed 
to  produce  it.  PRRs,  properly  staffed  and  conducted,  will  result  in  both  Government 
and  contractor  benefits.  Government  policy  and  procedures  on  conducting  PRRs 
are  contained  in  DoD  Instruction  5000.38  (reference  (j)). 


TIMELINE 
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A  transition  plan,  which  is  a  comprehensive  management  plan  describing  all  production- 
related  activities  that  must  be  accomplished  during  design,  test,  and  low  rSte  initial 
production,  is  needed  to  ensure  a  smooth  transition  from  development  to  full  rate 
production.  To  be  effective,  the  transition  plan  should  be  available  before  the  start  of  FSD 
and  updated  regularly  so  that  low  rate  production  can  be  initiated  at  minimal  risk. 
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CHAPTER  7 


INTRODUCTION  FOR 
FACILITIES  AND  CAPITAL  INVESTMENT 
CRITICAL  PATH  TEMPLATES 


Three  templates  are  provided  in  this  section.  The  first,  Modernization,  is  based  on  DoD’s 
new  Industrial  Modernization  Incentive  Program  (IMIP)  that  permits  profits  to  increase  as 
modernization  activities  reduce  costs  to  produce.  The  second,  Factory  Improvements,  is  an 
outline  of  an  electronics  factory  that  contains  the  equipment  required  to  implement  a  low 
risk  manufacturing  operation.  The  third,  Productivity  Center,  is  a  method  for  upgrading  the 
skills  of  personnel  using  the  new  equipment  and  processes  on  the  factory  floor. 
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TEMPLATE 


MODERNIZATION* 


AREA  OF  RISK 


Current  approaches  to  Government  contracting  fundamentally  inhibit  industry  investments 
to  modernize.  Why?  Profits  are  a  fixed  percent  of  the  cost  to  produce.  See  figure  7-1 .  The 
rate  of  modernization  is  low  because  profits  go  down  as  costs  to  produce  go  down.  The 
capital  to  invest  in  modernization  activities  is  not  available  in  Government  business.  Why 
modernize?  Increased  productivity  reduces  costs  to  produce.  The  defense  industrial  base 
surge  capability  is  improved.  U.S.  industry’s  position  in  the  international  marketplace  has 
improved.  The  increased  market  improves  the  U.S.  balance  of  payments  and  produces 
more  jobs.  Automation  improves  quality.  The  talent,  material,  and  computer  software 
required  to  implement  the  design  and  manufacturing  fundamentals  for  reliable  products  are 
made  possible  by  increased  capital,  and  reduce  the  risk  of  transitioning  from  development 
into  production. 

OUTLINE  FOR  REDUCING  RISK 

•  The  DoD  IMIP  permits  profits  to  increase  as  costs  to  produce  decrease.  This 
provides  additional  capital  that  is  available  to  increase  the  rate  of  modernization  that 
increases  productivity  and  further  reduces  production  costs,  and  thus  overall  costs 
to  acquire  defense  material.  See  figure  7-2.  The  objective  is  to  increase  the  rate  of 
modernization. 

•  Single  product  incentives  are  considered,  when  appropriate.  These  incentives  result 
in  contractor  proposals  for  major  productivity  enchancements,  limited  overall  factory 
modernization,  and  large  unit  cost  savings.  Unit  cost  savings  examples  (using  1 982 
dollars)  are  as  follows: 
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•  Multiple  product  incentives  are  considered,  when  appropriate.  These  incentives 
result  in  contractor  proposals  for  major  product-oriented  productivity  enhancements 
and  factory  modernization  improvements.  An  example  of  results: 

—  Savings:  initial  investment  =$  70,000,000 
estimated  savings  =  430,000,000 

—  Modernization  improvements:  automated  material  handling,  automated 
assembly  of  cables  and  harnesses,  and  automated  printed  wiring  assembly 
station. 

•  The  multiple  product,  single  DoD  focal  point  concept  is  utilized.  When  a  factory 
deals  with  a  single  DoD  focal  point  as  the  customer  for  all  its  products  and  profits 
increase  as  costs  to  produce  decrease,  modernization  of  the  DoD  industrial  base 
may  take  care  of  itself. 

•  Modernization  activities  are  checked  carefully  against  their  impact  on  life  cycle  cost, 
i.e.,  product  quality. 

•  Contractor  funding  of  modernization  activities  is  preferred  by  the  Government,  and 
resultant  savings  are  shared  by  the  contractor  and  the  Government.  The  contractor  s 
investments  are  guaranteed  by  the  Government,  when  appropriate. 

•  Modernization  activities  are  flowed  down  to  subcontractors  and  suppliers,  to  accrue 
the  greatest  benefits. 

•  All  defense  materials,  not  just  weapon  systems,  are  considered  candidates  for 
modernization  activities. 


TIMELINE 


Factory  modernization  is  essential  to  cost-effective  production  of  today’s  sophisticated 
weapon  systems.  Modernization  activities  primarily  are  oriented  to  support  all  of  the  factory’s 
product  lines.  However,  there  may  be  program-related  activities.  In  these  cases,  detailed 
planning  is  done  early  enough  to  influence  the  design,  as  appropriate  and  required. 
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FACTORY1 

IMPROVEMENTS 


AREA  OF  RISK 

Many  equipment  failurss  in  the  field  can  be  attributed  to  excessive  manual  assembly  and 
test  operations  in  the  manufacture  of  assemblies.  In-plant  failures  from  manual  errors  in 
assembly  and  test  contribute  to  excessive  rework  and  repair  costs  (i.e.,  the  hidden 
factory”).  These  risk  areas  increase  production  and  life  cycle  costs  and  result  in  major 
schedule  risks.  These  risks  are  acute  particularly  during  the  transition  from  development  to 
production.  The  use  of  semiautomatic  equipment  in  electronics  manufacturing  is  essential 
in  reducing  these  risks.  This  template  illustrates  an  optimum  facility  for  electionics 
assembly  and  test  using  available  “off-the-shelf”  electronics  manufacturing  equipment. 


OUTLINE  FOR  REDUCING  RISK 

•  Incoming  inspection  and  automatic  kit  preparation  ensure  that  high  quality  and 
correct  components  are  used  on  the  assembly  line. 

—  Typically,  an  80  percent  reduction  in  component  defects  can  be  achieved. 

—  Exhibit  7-1 .  generically  illustrates  an  example  of  incoming  inspection  and  kit 
preparation  areas. 

•  Semiautomatic  and  fully  automated  circuit  board  assembly  techniques  increase 
productivity  and  minimize  assembly  and  workmanship  defects. 

—  Typically,  a  2:1  reduction  in  defect  rates  can  be  achieved. 

—.Exhibit  7-2.  generically  illustrates  an  example  of  a  circuit  board  assembly  and 
test  area. 
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•  Semiautomatic  assembly  and  test  techniques  maximize  productivity  and  minimize 
workmanship  defects  on  electronic  assemblies. 

—  Typically,  a  3:1  improvement  in  productivity  can  be  achieved. 

—  Exhibit  7-3.  generically  illustrates  an  example  of  an  electronics  subassembly 
and  test  area. 

•  One  hundred  percent  piece  part  inspection  of  electronic  parts  reduces  risk,  is  cost- 
effective,  and  should  be  a  routine  operation  in  incoming  inspection. 

•  A  productivity  center  for  personnel  training  and  development  of  any  equipment 
integration  minimizes  the  risk  of  unforeseen  throughput  problems. 

•  Computer-assisted  functions  include  a  data  interface  between  the  design  and 
operations  management  functions. 

•  Each  assembly,  test,  and  inspection  station  should  have  computer-aided  data  entry 
capability. 


TIMELINE 


The  use  of  state-of-the-art  factory  equipment  can  prevent  many  common  workmanship 
errors.  The  type  of  facility  planned  for  the  manufacture  of  the  end  item  product  should  be 
identified  during  engineering  development,  and  should  be  evaluated  periodically  from 
development  until  full  rate  production  is  achieved. 
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EXHIBIT  7-1.  INCOMING  INSPECTION  AND  KITTING 


EXHIBIT  7-2.  PRINTED  CIRCUIT  BOARD  ASSEMBLY  AND  TEST 


ELECTRONICS  ASSEMBLY  AND  TEST 
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TEMPLATE 


PRODUCTIVITY 


AREA  OF  RISK 


The  use  of  changing  technology  on  the  factory  floor  without  qualified  personnel  can  be 
counterproductive,  lowering  or  eliminating  the  productivity  gains  anticipated  from  the  capital 
investment  in  modernization  and  factory  improvement.  Thus,  maintaining  a  stable  labor 
force  as  new  technology  is  introduced  on  the  factory  floor  is  a  risk  area.  This  risk  area  is 
amplified  with  the  introduction  of  new  “state-of-the-art”  products  that  are  typical  of  today’s 
military  weapon  systems.  Training  and  maintaining  the  skill  of  the  labor  force,  therefore, 
presents  a  significant  risk  in  the  transition  to  production.  A  productivity  center  that  updates 
the  skills  of  the  work  force  and  provides  orientation  training  for  new  product  lines  is  a 
catalyst  for  maintaining  a  well-trained  labor  force.  This  template  provides  a  framework  for 
an  effective  productivity  center. 


OUTLINE  FOR  REDUCING  RISK 

•  Productivity  center  includes  an  apparatus  lab  that  contains  the  equipment  and 
technologies  that  represent  the  actual  facility  producing  a  product. 

—  Use  of  the  apparatus  lab  includes  simulation  of  production  equipment  hardware 
and  end  item  defects. 

—  The  apparatus  lab  evaluates  new  processes  or  process  changes  before 
introduction  at  the  main  facility.  This  technique  ensures  that  any  change  to 
existing  procedures  will  not  affect  adversely  normal  production  flow. 

•  Productivity  center  includes  a  learning  center  for  classroom  instruction  for  updating 
the  skills  of  manufacturing  personnel. 
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•  Training  system  is  flexible  and  individual  performance  oriented. 

—  Sixty  percent  is  “hands  on”  training  in  apparatus  lab. 

—  Forty  percent  is  formal  classroom  instruction. 

—  Attention  is  given  to  skill  assessment  and  the  motivation  aspects  of  worker 
retraining. 

•  Typical  training  courses  include  the  following: 

—  Product  orientation. 

—  Manufacturing  facility  orientation. 

—  Electronics  manufacturing  and  test  operations  and  procedures. 

—  Numerical  control  machine  operations. 

-CAM. 

—  Diagnostics  for  troubleshooting  and  repair  (system  level). 

—  Microprocessor  troubleshooting  techniques. 

—  Computer  technology. 

TIMELINE 


A  productivity  center  provides  an  “off-line"  capability  to  evaluate  manufacturing  techniques 
for  worker  retraining  for  production  line  improvements.  As  new  technology,  equipment, 
manufacturing  processes,  or  test  procedures  are  identified  for  the  efficient  production  of  a 
specific  product,  personnel  must  be  trained  to  perform  these  new  tasks.  Manufacturing 
engineering  concurrent  with  design  engineering  will  identify  these  tasks  during 
development,  and  additional  tasks  will  be  identified  until  rate  production  has  been  achieved. 
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CHAPTER  8 


INTRODUCTION  FOR  LOGISTICS  CRITICAL  PATH  TEMPLATES 


The  primary  purpose  of  the  acquisition  process  is  to  field  weapon  systems  and  equipment 
that  not  only  perform  their  intended  functions,  but  are  ready  to  perform  these  functions 
when  called  on,  and  to  do  so  over  and  over  again  without  unplanned  maintenance  and 
logistics  efforts.  However,  numerous  examples  abound  when  new  systems,  when  fielded, 
do  not  achieve  readiness  levels  to  meet  service  needs,  necessitating  engineering  and 
manufacturing  changes  as  well  as  additional  equipment,  spares,  and  maintenance 
resources,  all  of  which  increase  cost  as  well  as  production  and  deployment  risk. 

The  templates  in  this  section  address  logistics  and  supportability  issues  that  contribute  to 
the  risk  of  transition  from  development  to  production.  Accordingly,  they  do  not  explicitly 
refer  to  all  integrated  logistics  support  (ILS)  elements  or  outline  a  total  strategy  for  ILS 
planning  and  management  in  the  acquisition  process.  These  elements  and  strategy  are 
covered  in  DoD  Directive  5000.39  (reference  (k))  and  Military  Service  implementing 
documents.  As  specified  in  reference  (k),  the  acquisition  manager  is  required  to  develop  an 
ILS  plan  that  successfully  coordinates  the  areas  addressed  in  this  logistics  section.  The 
logistics  elements  and  supportability  issues  and  their  requirements,  outlined  in  this  section, 
represent  those  that  have  been  particularly  difficult  and  destabilizing,  and  require  special 
attention.  Therefore,  the  implementation  of  the  concepts,  procedures,  and  techniques 
discussed  in  this  section  will  reduce  significantly  the  risk  of  transition  from  development  to 
production  and  deployment. 
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Logistics  Support  Analysis  (LSA)  is  used  throughout  the  acquisition  process  to  evaluate 
design  approaches  and  alternative  support  concepts  to  achieve  system  readiness  and 
support  objectives,  and  to  develop  detailed  design  of  the  support  system  and  requirements. 
Weapon  system  programs  that  have  either  delayed  the  application  of  LSA  or  have  not 
integrated  it  effectively  into  the  design  analysis  process  are  headed  for  trouble.  The  result  is 
supportability  deficiencies  that  increase  costs  and  require  additional  engineering  changes 
to  correct  these  deficiencies  late  in  the  development  and  production  process. 


OUTLINE  FOR  REDUCING  RISK 

•  Design  objectives  and  development  of  design  options  to  achieve  readiness  and 
supportability  objectives  are  required  by  the  engineering  statement  of  work  (SOW). 

•  LSA  is  integrated  into  the  design  process  to  determine  design  impact  on  support. 

•  The  LSA  process  has  identified  high  leverage  subsystem  and  component  reliability 
and  maintainability  efforts  needed  to  achieve  readiness  and  deployment  objectives. 

•  Quantitative  logistics  and  supportability  requirements  are  given  explicit  weight  in 
source  selection. 

•  LSA  data  is  derived  from  the  same  source  data  used  by  design  and  test  engineering. 

•  The  engineering  disciplines  have  an  “agreed  to”  methodology  for  quantifying 
readiness  and  supportability  design  impacts. 
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•  Disposition  of  LSA-identified  cost  and  performance  drivers  are  coordinated  with  the 
users  to  permit  meaningful  tradeoffs. 

•  Adequate  funding  and  technical  manpower  are  programed  to  perform  LSA  analyses 
required  during  the  concept  demonstration  and  validation  phase  and  followup. 


TIMELINE 
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The  LSA  is  begun  early  in  the  development  process  to  explicitly  address  supportability  and 
support  requirements  throughout  the  design,  development,  and  production  process. 
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AREA  OF  RISK 


Weapon  systems  and  support  systems  must  be  designed  with  as  complete  an 
understanding  as  possible  of  user  manpower  and  personnel  skill  profiles.  A  mismatch 
yields  reduced  field  reliability,  increased  equipment  training,  technical  manual  costs,  and 
redesign  as  problems  in  these  areas  are  discovered  during  demonstration  tests  and  early 
fielding.  Discovery  of  increased  skill  and  training  requirements  late  in  the  acquisition 
process  creates  a  difficult  catchup  problem  and  often  leads  to  poor  system  performance. 


OUTLINE  FOR  REDUCING  RISK 

•  Manpower  and  skill  requirements  are  based  on  formal  analysis  of  previous 
experience  on  comparable  systems  and  maintenance  concepts.  This  is  done  under 
contract  during  the  preconceptual  through  validation  phase. 

•  RFPs  reflect  the  required  priority  for  reducing  manpower  quantities  or  skill 
requirements.  This  is  backed  up  by  detailed  descriptions  of  current  and  projected 
manpower  skill  resources  and  shortfalls.  This  data  includes  specific  information  on 
current  maintenance  and  operator  performance  and  realistic  manpower  costs  on 
similar  fielded  systems. 

•  Arrangements  are  made  for  the  contractor  to  observe  maintenance  in  the  field  to 
gain  appreciation  for  capabilities  and  constraints. 

•  Manpower  cost  factors  used  in  design  and  support  tradeoff  analyses  take  into 
account  costs  to  train  or  replace  experienced  personnel,  as  well  as  billet  and  true 
overhead  costs. 
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Manpower  and  skill  requirements  are  established  early  in  the  conceptual  phase  and  are 
considered  as  prime  design  considerations  during  development.  They  are  addressed 
specifically  during  LSA,  and  tradeoffs  in  design  are  made  to  minimize  their  requirements. 
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TEMPLATE 


i  SUPPORTANIE 
fTES.1T  EQUIPMENT 


AREA  OF  RISK 

Weapon  system  supportability  is  dependent  on  reliable  and  maintainable  support  and  test 
equipment  that  can  be  deployed  with  the  prime  system.  However,  the  development, 
production,  and  fielding  of  this  equipment  have  been  a  common  source  of  risks  in  terms  of 
increased  costs,  schedule  delays,  and  poor  performance  and  readiness  for  fielded 
systems.  The  more  significant  causes  of  this  risk  are:  (1)  delayed  identification  of  support 
equipment  requirements;  (2)  design  and  development  of  software  intensive  support 
equipment  before  design  stability  of  the  system  it  supports;  (3)  underestimation  of  software 
requirements  and  development  costs;  and  (4)  failure  to  apply  sound  engineering, 
manufacturing,  and  management  disciplines  to  the  design,  development,  test,  and 
production  of  support  and  test  equipment. 


OUTLINE  FOR  REDUCING  RISK 

•  Identification  of  support  equipment  needs,  as  part  of  the  LSA  process,  is  initiated  as 
early  in  development  as  prime  system  concept  permits. 

•  Test  equipment  performance  specifications  include  criteria  for  fault  detection, 
isolation,  and  false  indications. 

•  Phased  contractor  support  is  utilized  to  allow  for  design  instability. 

•  Test  equipment  performance,  procedures,  and  software  verification  and  validation 
are  completed  before  contractor  support  termination. 

•  Upward  compatibility  is  specified  between  BIT  and  intermediate,  deKot,  and  factory 
levels  of  support  equipment. 
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•  Support  and  calibration  requirements  for  test  equipment  are  included  in 
development  and  production  contracts. 

•  Estimated  costs  of  test  program  set  (TPS)  development  are  based  on  comparable 
equipment  development  and  are  funded  fully. 

•  Support  and  test  equipment  is  evaluated  during  formal  contractor  maintainability 
demonstrations  and  in  operational  tests. 


TIMELINE 
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Support  and  test  equipment  design,  test,  production,  and  supportability  follow  the  same 
processes  outlined  in  this  Manual  for  the  prime  equipment. 
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TEMPLATE 


:  TRAINING 

i  MATERIALS  ANffi 
i  cmiraiuiciuir 


AREA  OF  RISK 

On  some  programs,  training  requirements  are  not  addressed  adequately,  resulting  in  great 
difficulty  in  operation  and  support  of  the  hardware.  Training  programs,  materials,  and 
equipment  such  as  simulators  may  be  more  complex  and  costly  than  the  hardware  they 
support.  Delivery  of  effective  training  materials  and  equipment  depends  on  the 
understanding  of  final  production  design  configuration,  maintenance  concepts,  and  skill 
levels  of  personnel  to  be  trained.  On  many  programs,  training  materials  and  equipment 
delivery  schedules  are  overly  ambitious.  The  results  include  poor  training,  inaccuracies  in 
technical  content  of  materials,  and  costly  redesign  and  modification  of  training  equipment. 


OUTLINE  FOR  REDUCING  RISK 

•  Contractors  are  provided  with  clear  descriptions  of  user  personnel  qualifications  and 
current  training  programs  of  comparable  systems,  to  be  used  in  prime  hardware  and 

training  systems  design  and  development. 

¥ 

•  Maintenance  tasks  identified  through  LSA  provide  the  data  base  used  in 
comprehensive  training  program  development  systems  (such  as  instructional 
systems  development  (ISD)). 

•  Computer-aided  techniques  are  used  for  configuration  control  to  ensure  consistency 
between  training  materials  and  equipment  and  the  systems  they  support. 

•  On-the-job  training  capability  is  incorporated  in  the  prime  equipment  design  as  a 
method  to  reduce  the  need  for  additional  training  equipment. 
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•  Complex  and  costly  training  equipment,  such  as  simulators,  is  scheduled  to  be 
produced  after  design  freeze  of  the  prime  equipment. 

TIMELINE 


Training  materials  and  equipment  must  match  maintenance  plans.  Equipment  built-in 
training  features  must  be  established  early  in  the  design  phase,  and  the  training  device 
design  must  reflect  stable  prime  equipment  design. 
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TEMPLATE 


H  MONEY 
PHASING 


AREA  OF  RISK 


SPAREST* 


Spares  are  a  troublesome  area  in  the  production  and  deployment  of  weapon  systems. 
Spares  and  repair  parts  often  do  not  meet  the  same  quality  and  reliability  levels  as  the 
prime  hardware.  Full  spares  provisioning  too  early  in  the  development  cycle,  when  there 
are  large  uncertainties  in  the  predicted  failure  rates  and  design  stability,  results  in  the 
procurement  of  unneeded  or  unusable  spares.  Inadequate  technical  and  reprocurement 
data  frequently  limits  competition,  acquisition  flexibility,  and  spares  manufacturing 
throughout  the  life  cycle  of  the  prime  systems.  Spares  thus  present  a  major  risk  of 
increased  acquisition  and  support  costs  and  reduced  readiness  of  fielded  systems. 


OUTLINE  FOR  REDUCING  RISK 

•  A  spares  acquisition  strategy  is  developed  early  in  FSD  to  identify  least  cost  options, 
including  combining  spares  procurement  with  production.  This  strategy  addresses 
spares  requirements  to  meet  FSD  testing  as  well  as  production  and  deployment. 

•  The  same  quality  manufacturing  standards  and  risk  reduction  techniques  used  for 
the  prime  hardware  are  used  in  the  spares  manufacturing  and  repair  process. 

•  Transition  from  contractor  to  Government  spares  support  is  planned  on  a  phased 
subsystem-by-subsystem  basis. 

•  Initial  spares  demand  factors  are  based  on  conservative  engineering  reliability 
estimates  of  failure  rates  (derived  from  comparability  analysis)  and  sparing  to 
availability  analytical  models.  These  factors  are  checked  for  reasonableness  at  the 
system  or  major  subsystem  level  against  laboratory  and  field  test  results  and 
documented  in  the  logistics  support  analysis  data  base. 
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•  Technical  and  reprocurement  data  is  validated  by  analysis  and,  when  possible,  by 
“proof  models,"  to  ensure  the  quality  of  the  spares  and  repair  parts  production 
process. 

•  Plans  for  developing  spares  procurement  and  manufacturing  options  to  sustain  the 
system  until  phaseout  are  considered  in  the  production  decision.  These  plans 
include  responsibilities  and  funding  for  configuration  management,  engineering 
support,  supplier  identification,  and  configuration  updates  of  factory  test  equipment 
to  the  current  fielded  configuration  of  the  produced  item. 


TIMELINE 
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Key  factors  in  the  risk  equation  are  operational  utilization,  spares  provisioning,  design 
stability,  adequacy  of  technical  and  reprocurement  data,  and  quality  of  spares 
manufacturing  and  repair  process. 
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TEMPLATE 


AREA  OF  RISK 

Technical  manuals  frequently  do  not  match  the  production  configuration  of  the  equipment 
supported.  The  manuals  are  difficult  to  read  and  understand.  These  deficiencies  cause 
delays  in  operational  testing,  low  readiness  rates,  increased  revisions  change  activity,  and 
increased  spares  and  data  costs. 


OUTLINE  FOR  REDUCING  RISK 

•  A  clear  delineation  of  Government  and  contractor  responsibilities  in  the 
development,  verification,  validation,  and  publication  of  technical  manuals  is  outlined 
in  the  ILS  plan. 

•  Automated  processes  (such  as  the  use  of  computer-aided  engineering  drawings  as 
illustrations)  are  used  in  technical  manual  preparation.  These  processes  are 
encouraged  by  RFP  requirements  and  evaluations  during  source  selection. 

•  The  LSA  process  analyzes  technical  options  for  portraying  information  including 
embedded  and  paperless  delivery. 

•  Maintenance  tasks  identified  through  the  LSA  process  provide  the  data  base  used  in 
technical  manual  development. 

•  Draft  manuals  are  validated  and  verified  before  final  preparation  and  publication. 
Equipment  availability  to  be  used  in  verification  and  validation  is  specified  in  the 
contract. 
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Automated  readability  analyses  are  used  to  verify  that  the  level  of  the  document 
matches  the  level  specified. 

The  milestone  schedule  includes  interim  manuals  for  initial  training. 


TIMELINE 
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The  development  of  technical  manuals  must  be  keyed  to  support  of  training  requirements, 
engineering  development  models,  equipment  evaluation,  initial  production  units,  and 
update  programs. 
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CHAPTER  9 

INTRODUCTION  FOR  MANAGEMENT  CRITICAL  PATH  TEMPLATES 


Our  free  enterprise  system  relies  heavily  on  the  law  of  supply  and  demand.  When  a 
supplier  has  the  capability  to  make  a  product  for  which  there  is  sufficient  consumer 
demand,  the  resources  of  both  the  supplier  and  the  customer  are  applied  to  ensure  that  the 
product  is  delivered  for  the  price  agreed  upon,  is  received  on  or  before  the  desired  date, 
and  performs  the  required  functions.  The  risk  drivers  in  this  process  include  the  quality  and 
experience  of  the  people  assigned  to  the  project.  More  specifically,  the  industry  supplier 
must  have  the  people  resources  to  design,  test,  and  produce  an  acceptable  end  item.  To 
ensure  that  customer  requirements,  and  any  necessary  changes  thereto  during  the 
acquisition  process,  are  communicated  effectively  to  the  supplier,  the  Government  also 
must  have  competent  people  resources  to  provide  clear  direction  and  evaluate  progress 
throughout  the  process. 

Without  adequate  numbers  of  competent  people  in  industry  and  Government,  there  is  an 
extremely  high  risk  of  having  an  unacceptable  product.  Although  material  and  time  are  very 
important  resources  requiring  effective  management,  people  are  the  key  to  a  successful 
program. 


9-1 


DoD  4245.7-M 


TEMPLATE 


I 


DESIGN  REF 
MISSION 
PROFILE 


DESIGN 

POLICY 


PARTS  AND 
MATERIALS 
SELECTION 


DESIGN 


OESIGN 

REQUIREMENTS 


OESIGN 

PROCESS 


SOFTWARE 

OESIGN 


|  OESIGN  FOR 
(  TESTING 


Ht 


BUILT-IN  TEST 


- 

1 

1 

COMPUTER-AIDED 
OESIGN  (CADI 

1 

1 

CONFIGURATION 

CONTROL 

OESIGN 

REVIEWS 


OESIGN 

RELEASE 


- ! - 

- 1 - 

TEST 

PRODUCTION 

I - 

|  FACILITIES 


1 

1 

1 

1 

FAILURE 

REPORTING 

SYSTEM 

■■ 

UNIFORM 

TEST 

REPORT 

SOFTWARE 

TEST 

PIECE  PART 
CONTROL 

1 

1 

1 

1 

UFE 

■■ 

■■■■ 

TEST  ANALYZE. 
AND  FIX  (TAAFt 

■ 

FIELD 

FEEDBACK 

SPECIAL  TEST 
EQUIPMENT  (STEl 

MANUFACTURING 

SCREENING 

LOGISTICS 

SUPPORT 

ANALYSIS 


MANPOWER 
ANQ  PERSONNEL 


SUPPORT  AWJ 
TEST  EQUIPMENT 


TRAINING  ANQ 
N  MATERIAL 
i  EQUIPMENT 


PERSONNEL 

REOUIREMENTS 


REQUIREMENTS 


TECHNICAL 

H  *SK 

ASSESSMENT 


_ I 

TRANSITION  PLAN 

_ 

_ 

_ 1 

— 

- 1 

AREA  OF  RISK 


One  of  the  first  tasks  for  the  newly  assigned  program  manager  is  the  development  of  an 
overall  acquisition  strategy.  Construction  of  the  program  acquisition  strategy  without  due 
consideration  to  the  manufacturing  elements  is  a  key  area  of  risk  to  the  capability  of  the 
industrial  base  meeting  the  schedule,  performance,  and  quality  desired  of  the  end  item.  If 
the  principal  contractors  do  not  know  what  is  expected  of  them  by  the  Government,  they  will 
be  uncertain  and  reluctant  to  make  the  proper  financial  and  personnel  resource  decisions 
necessary  for  facilitization,  industrial  modernization,  labor  commitments,  subcontractor  or 
vendor  structure,  and  foreign  and  domestic  technology  and  production  sharing 
agreements.  Inadequate  and  unnecessarily  imprecise  production  planning  information 
increases  program  risk  to  the  contractors  and  adds  delay  and  indifference  to  industrial 
market  participation  in  the  program.  Resulting  inefficiencies  will  increase  substantially 
production  and  support  costs. 


OUTLINE  FOR  REDUCING  RISK 

•  A  manufacturing  strategy  as  specified  by  DoD  Directive  4245.6  (reference  (h))  is 
outlined  by  the  program  manager  as  part  of  the  initial  acquisition  strategy.  The 
manufacturing  strategy  is  refined  progressively  during  the  program's  conceptual 
phase  so  that  a  sound,  comprehensive  manufacturing  approach  is  available  for 
dissemination  with  the  solicitations  for  the  development  effort. 

•  Demands  on  the  industrial  base  will  be  discernable  readily  from  stated  inventory 
objectives,  operational  capability  dates,  initial  production  requirements,  delivery 
profiles,  and  production  surge  requirements. 
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•  Maintenance,  logistics,  mobilization,  and  surge  planning  information  enables 
assessing  the  demands  on  production  capacity  from  spares  and  test  and  support 
equipment  requirements. 

•  Use  of  strategic  and  critical  materials  and  vendor  manufacturing  capabilities  is 
projected,  including  offshore  requirements. 

•  Critical  manufacturing  technologies  needed  to  efficiently  produce  the  concept  and 
the  design  are  identified  and  pursued  through  appropriate  RDT&E  projects. 

•  Peculiar  system  and  component  manufacturing  test  equipments  are  scheduled  for 
development  and  use. 

•  The  contracting  scheme  is  compatible  with  program  risk  and  needed  levels  of 
Government  visibility  and  control. 

•  The  contractors  are  aware  fully  of  Government  plans  for  dual  sourcing  and 
“breakout”  of  Government-furnished  equipment  so  that  rights  in  data  and 
technology  transfer  issues  are  resolved  expeditiously.  Procurement  of  necessary 
technical  data  is  an  integral  part  of  the  development  effort. 

•  The  Government  manufacturing  strategy  is  translated  readily  into  contractor 
production  and  transition  planning  documents  that  convincingly  show  the 
contractors’  appreciation  of  and  capability  to  respond  to  the  magnitude  and 
complexity  of  the  manufacturing  effort  and  their  willingness  to  participate  in 
mobilization,  surge,  and  productivity  enhancement  projects. 

•  Production  matters  are  weighted  heavily  in  engineering  development  source 
selection  evaluations  and  the  contractors  are  so  informed. 

TIMELINE 


A  manufacturing  strategy  should  be  developed  at  the  initiation  of  program  development  to 
reduce  risk  while  meeting  cost,  schedule,  performance,  and  quality  of  the  production  items. 
As  development  progresses,  the  manufacturing  strategy  should  be  refined  and  updated  so 
that  a  sound  manufacturing  approach  is  in  place  at  the  start  of  production. 


9-3 


DoD  4245.7-M 


TEMPLATE 


AREA  OF  RISK 
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It  is  a  common  practice  in  both  industry  and  Government  for  program  managers  to  be 
supported  by  a  small  number  of  key  staff  personnel  collocated  in  the  program  office  and  by 
a  large  number  of  functional  area  experts  who  provide  their  support  using  a  matrix 
management  approach.  Contractor  program  managers  may  lack  the  experience  to 
orchestrate  the  entire  effort  from  drawing  board  to  finished  product.  Government  program 
managers  may  likewise  lack  acquisition  experience  and  proven  leadership  ability,  and  tour 
lengths  are  often  too  short  to  see  the  program  through  to  completion.  Engineering  and 
manufacturing  talent  may  lack  critical  continuity  and  corporate  knowledge.  For  example, 
design  engineering  may  be  left  to  recent  college  graduates  because  the  more  experienced 
design  engineers  have  been  promoted  to  new  fields  of  endeavor.  Functional  support 
personnel  are  also  in  the  critical  path,  and  the  recruitment,  training,  and  retention  of 
competent,  experienced  personnel  may  not  be  a  continuing  corporate  objective.  History 
has  proven  that  those  programs  for  which  Government  or  industry  top  managers  only  gave 
lip  service  to  the  precept  that  states  “people  are  our  most  important  resource’’  have 
suffered  and  often  failed. 


OUTLINE  FOR  REDUCING  RISK 

•  Career  progressions  are  defined  for  prospective  program  managers,  and  available 
formal  training  such  as  the  Defense  Systems  Management  College  and  informal 
training  such  as  training  with  industry  programs  (for  DoD  personnel)  are  used. 

•  Program  manager  tours  are  extended  and  stabilized,  particularly  in  the  Department 
of  Defense,  and  civilian  program  managers  are  used  in  the  Department  of  Defense 
on  a  selected  basis.  Stability  considerations  argue  strongly  against  changing 
program  managers  and  key  staff  and  functional  support  personnel  at  major  program 
milestones. 
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•  A  program  manager  never  is  assigned  more  than  one  major  program. 

•  The  use  of  matrix  management,  a  proven  concept,  is  coupled  with  as  much 
collocation  of  key  functional  support  personnel  as  practical. 

•  Line  managers  are  involved  in  the  recruitment,  training,  and  retention  of  key 
technical  personnel  rather  than  delegating  all  such  responsibility  to  the  personnel 
support  organization.  To  provide  DoD  line  managers  with  greater  control  over 
personnel  functions,  innovative  techniques,  such  as  the  Civil  Service  experiment 
being  conducted  at  the  Naval  Weapons  Center  (NWC),  China  Lake,  and  Naval 
Ocean  Systems  Center  (NOSC),  San  Diego,  are  considered. 

•  Personnel  with  production  experience  are  critical  particularly  in  Government 
organizations  because  manufacturing  operations  usually  are  contracted  with 
industry.  Career  development  and  training  programs  with  a  production  orientation 
are  supported  zealously  by  the  Military  Services,  and  program  managers  ensure 
that  their  personnel  attend  or  have  commensurate  experience. 


TIMELINE 


Personnel  resources  are  the  key  determinant  of  success  or  failure  throughout  the  life  cycle 
of  any  program.  To  recruit,  train,  and  retain  the  people  necessary  to  ensure  success,  it  is 
essential  that  Government  and  industry  couple  effective  management  and  sound 
leadership  during  every  program  phase,  including  the  transition  from  development  to 
production. 
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TEMPLATE 


AREA  OF  RISK 


DAT/S - 
REQUIREMENTS! 


The  Government  asks  for  too  much  technical  data  in  their  procurements,  which  increases 
the  risk  of  cost  overruns.  Redundant  data  also  may  be  procured  by  different  Government 
functional  organizations  and  the  program  office  that  did  not  coordinate  their  data 
requirements  before  contract  definition.  Often,  this  is  a  direct  result  of  using  a  boilerplate  list 
of  data  requirements  when  the  request  is  submitted  by  the  various  Government  offices 
responsible  for  the  procurement.  It  is  estimated  that  direct  costs  for  data  range  from  6  to  20 
percent  of  contracts,  not  including  the  overhead  costs  and  the  cost  to  the  Government  to 
process,  review,  and  manage  the  data.  A  corollary  problem  is  the  degree  to  which  any 
potentially  useful  data  is  evaluated  and  introduced  into  the  decision  making  process.  On 
the  one  hand,  too  much  data  is  required  and,  on  the  other,  not  enough  data  is  used  for 
better  program  control.  Control  of  data  requirements  has  been  sporadic  at  best  and,  even 
though  the  problem  of  poor  data  management  has  been  identified  in  various  studies  over 
the  past  20  years,  it  receives  little  emphasis  because  of  little  top  level  commitment. 


OUTLINE  FOR  REDUCING  RISK 

•  All  procurement  data  requirements  are  reviewed  using  an  effective  data  review 
board  before  contract  award,  to  ensure  that  the  data  received  will  satisfy  the 
Government’s  needs,  is  in  a  format  suitable  for  customer  use,  and  is  not  redundant. 

•  An  integrated  data  management  system  is  established  both  in  Government  and 
industry  for  each  major  procurement.  The  objective  is  to  tailor  the  technical  data 
requirements  to  the  needs  of  each  program. 

•  Electronic  data  transfer  is  used.  Pertinent  data  required  by  the  Government  can  be 
requested  by  accessing  the  customer  data  base.  The  requested  data  can  then  be 
exercised  in  the  Government’s  data  base  to  extract  the  required  information. 
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•  The  data  requirements  for  a  major  program  are  reviewed  at  a  level  high  enough  to 
ensure  that  redundant  data  is  not  being  requested  by  the  different  disciplines  within 
the  program  office  and  its  functional  support  organizations. 

•  Technical  data  libraries  are  established  for  ease  of  data  retrieval,  and  the  data  is 
kept  current. 

•  Data  requirements  are  reviewed  during  each  phase  of  the  program  to  ensure  that 
data  being  procured  meets  the  needs  of  that  particular  program  phase. 

•  Data  is  procured  using  well-defined  data  requirements  lists,  reasonable  cost 
estimates,  and  realistic  schedules. 


TIMELINE 


^^^^PROGRAM  PHASE 
TEMPLATE  ACTIVIH^^^^ 

DEPLOY* 

JMSNS  1  N  HI  A  IIIB  ment 

_ A _ A _ A _ A _ 

Management 

Manufacturing  Strategy 

Personnel  Requirements 

2— 

j  Technical  Risk  Assessment 

I  Production  Breaks 

1 

' 

Useful  data,  properly  applied  during  the  decision  making  process,  will  ensure  that  the 
system  being  procured  meets  all  the  technical  requirements  and  that  the  necessary 
reprocurement  information  is  available  when  needed.  An  integrated  data  management  plan 
developed  at  the  start  of  the  program  and  approved  at  the  appropriate  management  level, 
should  lay  out  the  technical  data  requirements  for  all  phases  of  the  program  to  reduce 
management  risks.  ' 
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TEMPLATE 


DESIGN  HEF 
MISSION 

profile 


PARTS  AND 
MATERIALS 
SELECTION 


DESIGN 

REVIEWS 


>{ 


1 

DESIGN 

1 

DESIGN 

"  REQUIREMENTS 

„  DESIGN 

PROCESS 

SOFTWARE 
'  OESIGN 

,  BUILT-IN  TEST 

OESIGN 

RELEASE 

f 

DESIGN 

ANALYSIS 


(JNIFOHM 

TEST 

REPORT 


DESIGN 

LIMIT 


CONFIGURATION  TEST.  ANALYZE 
CONTROL  I  AND  FIX  ITAAFI 


TRANSITION  PLAN 


FAILURE 

REPORTING 

SYSTEM 


MANUFACTURING! 
PLAN  I 


PIECE  PART 
CONTROL 


DEFECT 

CONTROL 


QUALIFY  MFG. 
PROCESS 


TOOL .] 
PLANNING  | 


JCOMPUTER-AlOEOj 
MFG.  (CAMt 


r  : - 

1 

I 

- 1 

TEST 

PRODUCTION 

FACILITIES 

LOGISTICS 

MANAGEMENT 

MODERNIZATION 


FACTORY 

IMPROVEMENTS 


LOGISTICS 

SUPPORT 

ANALYSIS 


SUPPOHT  ANO 
TEST  EQUIPMENT 


TRAINING  ANO 
MATERIAL 
EQUIPMENT 


H  SPARES- 


PERSONNEL 

REQUIREMENTS 


TECHNICAL 

RISK 

ASSESSMENT 


AREA  OF  RISK 

The  track  record  of  major  defense  systems  acquisitions  has  been  poor  over  the  past 
several  years,  as  manifested  by  the  length  of  the  acquisition  cycle,  the  unsatisfactory  levels 
of  effectiveness,  and  the  pressure  to  reduce  life  cycle  costs.  In  spite  of  numerous  attempts 
to  improve  the  management-oriented  Defense  System  Acquisition  Review  process,  the 
lack  of  consistent  and  predictable  success  has  resulted  in  renewed  interest  in  upgrading 
the  process  by  an  infusion  of  technical  discipline.  The  1981  DoD  Acquisition  Improvement 
Program  not  only  identified  the  root  cause  of  acquisition  problems  to  be  “uncertainty”  but 
also  called  for  increasing  DoD  efforts  to  quantify  risk  and  for  expanding  the  use  of  budgeted 
funds  to  deal  with  uncertainty.  Since  risk  and  the  degree  of  uncertainty  are  synonymous 
and  directly  proportional  to  the  seriousness  of  the  acquisition  problems  faced  by 
Government  and  industry  program  managers,  why  have  many  years  of  alleged  emphasis 
on  technical  risk  assessment  achieved  so  few  results?  It  must  be  concluded  that 
management  ignorance  of  technical  risk  assessment  is  itself  a  major  source  of  risk  in  the 
transition  from  development  to  production. 


OUTLINE  FOR  REDUCING  RISK 

•  Technical  risk  management  is  specified  as  a  contractual  requirement,  and  early 
implementation  in  the  development  process  is  required. 

•  All  areas  of  risk  are  identified  as  early  as  possible  in  the  development  cycle.  A 
specific  set  of  tracking  indicators  is  determined  for  each  major  technical  element 
(design,  test,  and  production)  as  well  as  for  cost  and  management. 

•  Plans  are  developed  to  track,  measure,  assess,  and  adjust  for  identified  risks  using 
a  disciplined  system  that  can  be  applied  by  managers  from  a  variety  of  positions 
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within  the  Government  and  the  contractor  organizations.  This  system  provides  a 
continuous  assessment  of  program  health  against  quantifiable  parameters. 

•  Risk  drivers  are  understood  adequately  by  contractors,  using  qualified  design  and 
production  engineers  knowledgeable  of  the  risk  drivers,  to  identify  and  reduce 
program  technical  risks. 

•  Technical  problems  are  highlighted  before  they  become  critical. 

•  Hasty  shortcuts  are  avoided,  mission  profiles  are  reviewed,  and  existing  analysis 
tools  are  used  while  implementing  the  technical  risk  assessment  system. 

•  Test  programs  are  structured  to  verify  that  high  risk  design  areas  have  been 
resolved. 


TIMELINE 


'  ~^^PR0GRAM  PHASE 

TEMPLATE  ACTIVITY'''^^ 

0EP10Y- 

JMSNS  1  11  MIA  1 11 B  ment 

_ _ A _ A _ A _ A _ 

Management 

Manufacturing  Strategy 

Personnel  Requirements 

Data  Requirements 

r 

-K - la 

- Z-l 

1  Production  Breaks  | 

1 

A  technical  risk  assessment  system  should  provide  all  levels  of  management  with  (1)  a 
disciplined  system  for  early  identification  of  technical  uncertainties,  (2)  a  tool  for 
instantaneous  assessment  of  current  program  status,  and  (3)  early  key  indicators  of 
potential  success  or  failure.  To  be  effective,  a  technical  risk  assessment  system  should  be 
initiated  at  the  start  of  the  program  and  function  throughout  the  development  and 
production  phases. 
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TEMPLATE 


AREA  OF  RISK 


Changes  in  production  schedule  range  from  reduced  delivery  rate  (stretchout)  to  a 
complete  shutdown  of  the  production  line  (production  break).  Stretchouts  and  production 
breaks  increase  both  technical  risk  and  cost.  Factory  space,  tooling,  and  equipment  are 
idled,  and  in  the  worst  case,  may  be  eliminated.  Publications  and  handbooks  lose  currency. 
Production  flow  is  interrupted  and  benefits  from  assembly  improvements  and  automation 
are  lost.  Experienced  manufacturing  and  engineering  personnel  are  either  reassigned  or 
dismissed.  Morale  suffers,  teamwork  is  less  apparent,  problem  identification  and  resolution 
become  much  more  difficult  to  reestablish,  and  production  efficiency  degrades  noticeably. 
Design  improvements  are  less  effective  and  less  timely.  Small  suppliers  and  vendors 
whose  orders  represent  much  larger  percentages  of  their  total  business  are  less  able  to 
adjust,  and  in  the  worst  case,  even  sole  source  suppliers  and  vendors  have  been  forced  out 
of  business. 


OUTLINE  FOR  REDUCING  RISK 

•  Experience  has  shown  that  the  classic  result  of  a  production  break  is  as  illustrated  in 
figure  9-1.  The  ideal  solution,  of  course,  is  never  to  permit  a  break  to  occur. 
However,  when  the  realities  of  the  budget  process  increase  the  potential  for  a 
Government-mandated  production  break,  understanding  the  impact  might  help  the 
arguments  for  softening  such  a  decision  or  preventing  it  from  being  made  at  all. 

•  The  loss  of  learning  that  often  includes  a  loss  of  process  capability  results  in  an 
overall  program  cost  increase  and  a  higher  quantity  of  units  produced  before  unit 
cost  reaches  the  value  it  would  have  been  without  a  break  in  production.  A 
significant  reduction  in  production  rate,  to  a  “misery  rate”  level,  has  similar  effects. 
To  prepare  a  case  for  modifying  a  production  break  decision,  use  the  following 
method  to  compute  the  cost  of  the  loss  of  learning  (see  figure  9-1.): 
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—  Determine  value  of  learning  for  improvement  before  the  break  or  stretchout. 

—  Determine  percentage  loss  of  learning  for  duration  of  break  or  stretchout  and 
compute  new  cost  of  first  unit  produced  after  break  or  return  to  original 
production  rate. 

—  Locate  the  new  point  for  initial  unit  cost  following  break/return  to  original 
production  rate.  This  point  will  correspond  to  the  same  quantity  along  the 
abscissa  that  existed  just  before  the  break/reduction  in  rate. 

—  Develop  the  new  forecast  learning  curve  for  the  continuation  of  production. 

—  Loss  of  learning  cost  is  the  difference  between  the  cost  of  producing  the 
quantity  of  units  following  the  break  or  stretchout  versus  the  cost  of  the  same 
quantity  without  the  break  or  stretchout. 

•  Use  of  multiyear  contracting  minimizes  the  risk  of  production  breaks  or  stretchout. 


figure  9*1.  Production  Brook  Impact  on  Learning  Curve 
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TIMELINE 


PHASE 

TEMPLATE  ACTIVITY - ^ 

OEPIQY- 

JMSNS  I  II  IIIA  IHB  MENT 

_ A _ A _ A- _ A _ 

Management 

Manufacturing  Strategy 

Personnel  Requirements 

Oata  Requirements 

Technical  Risk  Assessment 

- T 

-I 

1 . 

IJ  E" . . 

— 

The  increase  in  production  efficiency  and  attendant  reduction  in  unit  cost  reflects  the 
benefits  of  an  uninterrupted  learning  curve,  that  is,  no  break  in  production,  starting  with 
initial  production  at  Milestone  IIIA. 
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ACRONYMS 


APL 

ATE 

Approved  Parts  List 

Automatic  Test  Equipment 

BIT  o 

o 

Built-In  Test 

D  4245.7-M 

<  <  Q  se 

O  O  O  O 

Computer-Aided  Design 

Computer-Aided  Manufacturing 

Critical  Design  Review 

Complimentary  Metal  Oxide  Semiconductor 

Do 

DPA 

DSARC 

DSB 

DT 

DT&E 

Department  of  Defense 

Destructive  Physical  Analysis 

Defense  Systems  Acquisition  Review  Council 

Defense  Science  Board 

Development  Test 

Development  Test  and  Evaluation 

ECP 

ESS 

Engineering  Change  Proposal 

Environmental  Stress  Screening 

FOT&E 

FRACAS 

FSD 

Follow-on  Operational  Test  and  Evaluation 

Failure  Reporting,  Analysis,  and  Corrective  Action  System 
Full-Scale  Development 

GAO 

General  Accounting  Office 

IES 

ILS 

IMIP 

IOT&E 

IPF 

ISD 

ITP 

Institute  of  Environmental  Sciences 

Integrated  Logistics  Support 

Industrial  Modernization  Incentive  Program 

Initial  Operational  Test  and  Evaluation 

Initial  Production  Funds 

Instructional  Systems  Development 

Integrated  Test  Plan 

JMSNS 

Justification  for  Major  System  New  Start 
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LOL 

Loss  of  Learning 

LSA 

Logistics  Support  Analysis 

MFHBF 

Mean  Flight  Hours  Between  Failure 

MTBF 

Mean  Time  Between  Failure 

MIL-HDBK 

Military  Handbook 

MIL-SPEC 

Military  Specification 

MIL-STD 

Military  Standard 

NOSC 

Naval  Ocean  Systems  Center 

NWC 

Naval  Weapons  Center 

OSD 

Office  of  the  Secretary  of  Defense 

OT 

Operational  Test 

PAT 

Production  Acceptance  Test 

PCB 

Printed  Circuit  Board 

PIN 

Particle  Induced  Noise 

PPBS 

Planning,  Programming,  and  Budgeting  System 

PRR 

Production  Readiness  Review 

RDT&E 

Research,  Development,  Test,  and  Evaluation 

RFP 

Request  for  Proposal 

SOW 

Statement  of  Work 

STE 

Special  Test  Equipment 

TAAF 

Test,  Analyze  and  Fix 

T&E 

Test  and  Evaluation 

TEMP 

Test  and  Evaluation  Master  Plan 

TPS 

Test  Program  Set 

TTL 

T ransistor/T ransistor  Logic 

USDR&E 

Under  Secretary  of  Defense  for  Research  and  Engineering 
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MEMORANDUM  FOR  SECRETARIES  OF  THE  MILITARY  DEPARTMENTS 
CHAIRMAN  OF  THE  JOINT  CHIEFS  OF  STAFF 
UNDER  SECRETARY  OF  DEFENSE  (POLICY) 

DIRECTOR,  DEFENSE  RESEARCH  AND  ENGINEERING 

ASSISTANT  SECRETARIES  OF  DEFENSE 

COMPTROLLER 

GENERAL  COUNSEL 

INSPECTOR  GENERAL 

DIRECTOR,  OPERATIONAL  TEST  AND  EVALUATION 
ASSISTANTS  TO  THE  SECRETARY  OF  DEFENSE 
DIRECTORS  OF  THE  DEFENSE  AGENCIES 

SUBJECT:  Total  Quality  Management  (TQM)  in  Acquisition  and  the 
Transition  from  Development  to  Production 


TQM  is  our  way-of-life  approach  to  conducting  the  defense 
acquisition  process.  In  keeping  with  this  philosophy,  I  have 
authorized  publication  of  the  attached  urgent  change  to  DoD 
4245. 7-M,  "Transition  From  Development  to  Production."  The 
purpose  of  this  change  is  to  guide  both  the  military  and  the 
private  sectors  of  the  defense  community  in  the  adoption  and 
use  of  TQM  principles. 

The  "transition"  or  "templates"  manual  covers  the  entire 
acquisition  process  and  is  already  a  TQM  document  in  concept. 
Certain  of  the  TQM  provisions  have  been  reemphasized  and 
aggregated  into  a  new  "TQM"  template  that  also  identifies  new 
and  proven  TQM  techniques  that  have  come  to  prominence.  The 
TQM  template  shall  be  used  in  conjunction  with  the  original 
manual,  September  1985,  pending  the  availability  of  a  more 
comprehensive  revision  in  12  to  18  months. 

TQM  is  applicable  to  all  DoD  activities  whether  concerned 
with  acquisition  or  not.  All  DoD  personnel  are  involved.  I 
enjoin  us  all  to  examine  our  functions  and  the  roles  we  play. 
Quality  must  be  uppermost  in  every  process.  The  execution  of 
each  of  our  jobs  must  add  value  to  the  products  we  make  and  the 
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Throughout  this  document  there  are  timelines  for  many 
template  activities  that  begin  and/or  end  between  two 
major  milestones.  In  such  cases,  the  timeline  is  depicted 
for  simplicity  purposes  as  beginning  and/or  ending  in  the 
middle  of  the  program  phase.  It  is  left  to  the  users  of  this 
document  to  determine  how  early  or  how  late  in  the  phase 
the  template  activity  begins  or  ends;  and  such  a  determin¬ 
ation  will  be  influenced  by  the  types  of  program,  the 
acquisition  plan,  and  the  best  judgment  of  experienced 
Government  and  industry  personnel. 

The  subsequent  pages  of  this  document  contain  all  the 
templates  generated  by  the  DSB  task  force  to  reduce  risk 
inherent  in  the  design,  test,  and  production  processes. 
Additional  templates  have  been  generated  as  a  result  of  a 
DoD  and  industry  wide  review.  Since  some  risk  is 
associated  with  funding,  facilities,  management  issues,  and 
the  transition  plan  for  design,  test,  and  production,  the  entire 
network  of  templates  is  arranged  in  a  sequence  considered 
logical  from  a  typical  program  manager's  viewpoint. 
Funding  is  presented  prominently  because  it  influences 
every  other  template  in  the  transition  document.  The  total 
network  of  critical  path  templates  is  shown  in  figure  1-2. 

In  figure  1-3,  the  time  phasing  associated  with  development 
of  each  of  the  templates  is  identified  as  the  program 
progresses  through  the  material  acquisition  cycle.  Program 
risk  is  introduced  when  a  particular  template  activity  is 
started  after  or  continued  beyond  the  timeline.  For  those 
less  familiar  with  the  DSARC  process  and  its  typical 
relationship  with  program  phasing,  the  conceptual  phase 
begins  after  the  justification  for  major  system  new  start 
(JMSNS)  is  approved.  Between  Milestones  I  and  II,  the 
demonstration/validation  phase  occurs  and  Milestone  II  is 
the  beginning  of  FSD.  The  production  phase  begins  at 
Milestone  MIA  (tooling,  long  lead  time,  and  pilot  production) 
notwithstanding  the  production  preparations  that  must  begin 
early  in  the  FSD  phase,  and  Milestone  IIIB  generally 
signifies  the  beginning  of  rate  production. 


Change  1  to  this  Manual  is  a  new  template  added  to  Chapter 
1  to  incorporate  Total  Quality  Management  (TQM).  In  the 
event  of  conflict  with  other  templates,  the  TQM  template 
takes  precedence. 


TEMPLATE 
APPLICABILITY  IS 
CORRELATED  WITH 
ACQUISITION  PHASES 
AND  MILESTONES 


NEW  DoD 
MANAGEMENT 
INITIATIVE  TAKES 
PRECEDENCE 
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Figure  1-2.  Critical  Path  Templates 


//First  Amendment  (Ch  1, 


2/13/89) 
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Transition  Plan 

Facilities 

Modernization 
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Logistics 

Logistics  Support  Analysis 
Manpower  and  Personnel 
Support  and  Test  Equipment 
Training  Materials  and  Equipment 
Spares 

Technical  Manuals 

Management 

Manufacturing  Strategy 
Personnel  Requirements 
Data  Requirements 
Technical  Risk  Assessment 
Production  Breaks 
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PROGRAM  RISK  IS  INTRODUCED  WHEN  A  PARTICULAR  TEMPLATE  ACTIVITY  IS  STARTEO  LATE 
OR  CONTINUES  BEYOND  THE  TIMELINE 


Figure  1-3.  Template  Timelines 

//First  Amendment  (Ch  1,  2/13/89) 
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CHAPTER  1 

INTRODUCTION  FOR  TQM  CRITICAL  PATH  TEMPLATE 


Since  publication  of  this  Manual  in  September  1985,  a  major  New  DoD  initiative  has 
been  instituted  called  TQM.  Change  1  to  this  Manual  provides  additional  guidance 
to  implement  the  philosophy  and  managerial  approach  involved  with  TQM  and 
consists  of  a  new  template  inserted  in  chapter  1 .  The  new  template  aggregates 
TQM  provisions  now  contained  in  the  Manual  by  highlighting  key  DESIGN,  TEST, 
and  PRODUCTION  template  activity  and  identifying  certain  advances  in  TQM 
methods  and  techniques  that  have  come  to  prominence.  Pending  a  more  extensive 
revision  to  this  Manual,  guidance  in  the  TQM  template  shall  take  precedence  in  the 
event  of  conflict  with  other  templates. 

TQM  is  the  disciplined  process  of  continuous  improvement  in  performance  at  every 
level  and  in  all  areas  of  responsibility  within  the  Department  of  Defense.  Improved 
performance  is  directed  toward  goals  assigned  to  cost,  schedule,  mission  need,  and 
operational  suitability.  Increasing  "user"  satisfaction  is  the  paramount  objective. 
Whereas  this  Manual  concentrates  on  the  industrial  process  concerned  with  the 
acquisition  of  materiel,  TQM  principles  are  applicable  equally  to  supporting  functions 
and  military  operations. 

TQM  was  approved  for  application  DoD-wide  by  the  Secretary  of  Defense  on  March 
30,  1988,  assigning  it  "top  priority."  The  DoD  posture  statement  on  quality  is 
reproduced  on  page  1-17.  On  August  30,  1988,  the  Under  Secretary  of  Defense  for 
Acquisition  issued  direction  to  implement  TQM  in  the  acquisition  process  and  called 
for  a  climate  in  both  Government  and  industry  that  would  foster  TQM  implementation. 

The  TQM  template  is  portrayed  at  the  top  of  the  template  network  in  figure  1-2, 
directly  supporting  the  product.  By  "product”  is  meant  systems,  equipments, 
hardware,  or  software,  and  supporting  services.  TQM  affects  everything  the 
Department  of  Defense  produces,  procures,  or  performs.  It  is  appropriate  to  all 
templates  and  nonacquisition  activities.  TQM  requires  professional  discipline  and 
commitment  from  both  the  Department  of  Defense  and  industry. 
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AREA  OF  RISK 

TQM  is  an  organized  process  of  continuous  improvement  by  private  defense  sectors 
and  DoD  activities  aimed  at  developing,  producing,  and  deploying  superior  materiel. 
The  primary  threat  to  reaching  and  sustaining  this  superiority  is  failure  to  manage  with 
a  purpose  of  constantly  increasing  the  instrinsic  quality,  economic  value,  and  military 
worth  of  defense  systems  and  equipments.  The  Armed  Forces  and  defense  industrial 
entities  may  not  attain  a  lasting  competitive  military  posture  and  long-term  competitive 
business  stature  without  a  total  commitment  to  quality  beginning  at  the  highest 
managerial  levels.  TQM  is  applicable  to  all  functions  concerned  with  acquisition  of 
defense  material,  supplies,  facilities,  and  services.  Being  satisfied  with  sub-optimum, 
short-term  goals  and  objectives  has  adverse  impacts  on  cost,  schedule,  and  force 
effectiveness.  A  short-term  approach  also  leads  to  deterioration  in  the  efficacy  of 
specific  products,  the  firms  that  produce  them,  and  the  industrial  base  overall.  Major 
risk  also  is  entailed  with  the  inability  to  grasp  and  respond  to  the  overriding 
importance  attached  to  quality  by  the  "customer"  or  user  activities. 

OUTLINE  FOR  REDUCING  RISK 

•  The  organization  has  a  "corporate  level"  policy  statement  attaching 
highest  priority  to  the  principles  of  TQM.  This  policy  statement  defines 
TQM  in  terms  relevant  to  the  individual  enterprise  or  activity  and  its 
products  or  outputs. 

•  The  corporate  policy  statement  is  supported  by  a  TQM  im¬ 
plementation  plan  that  sets  enduring  and  long  range  objectives,  lists 
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criteria  for  applying  TQM  to  new  and  on-going  programs,  provides 
direction  and  guidance,  and  assigns  responsibilities.  Every  employee 
at  each  level  plays  a  functional  role  in  implementing  the  plan. 

•  All  personnel  are  given  training  in  TQM  principles,  practices,  tools,  and 
techniques.  Importance  is  placed  on  self-initiated  TQM  effort. 

•  TQM  effort  begun  in  the  conceptual  phase  of  the  acquisition  cycle  is 
vitally  concerned  with  establishing  a  rapport  between  the  producer 
and  the  user  or  customer  and  a  recognition  of  the  latter's  stated 
performance  requirements,  mission  profiles,  system  characteristics,  and 
environmental  factors.  Those  statements  are  translated  into  meas¬ 
urable  design,  manufacturing,  and  support  parameters  that  are 
verified  during  demonstration  and  validation.  Early  TQM  activity  is 
outlined  in  the  Design  Reference  Mission  Profile  template  and  Design 
Requirements  template.  The  Trade  Studies  template  is  used  to  identify 
potential  characteristics  which  would  accelerate  design  maturity  while 
making  the  design  more  compatible  with  and  less  sensitive  to 
variations  in  manufacturing  and  operational  conditions. 

•  Design  phase  TQM  activity  is  described  in  the  Design  Process  template. 
Key  features  enumerated  include:  design  integration  of  life-cycle 
factors  concerned  with  production,  operation,  and  support;  availability  of 
needed  manufacturing  technology;  proof  of  manufacturing  process; 
formation  of  design  and  design  review  teams  with  various  functional  area 
representation;  and  use  of  producibility  engineering  and  planning  to 
arrive  at  and  transition  a  producible  design  to  the  shop  floor  without 
degradation  in  quality  and  performance.  The  Design  Analysis  template  and 
Design  Reviews  template  provide  guidance  in  identifying  and  reducing 
the  risk  entailed  in  controlling  critical  design  characteristics.  Both  hardware 
and  software  are  emphasized  (reference  the  Software  Design  template  and 
Software  Test  template).  A  high  quality  design  includes  features  to 
enhance  conducting  necessary  test  and  inspection  functions  (reference  the 
Design  for  Testing  template). 

•  An  integrated  test  plan  of  contractor  development,  qualification,  and 
production  acceptance  testing  and  a  test  and  evaluation  master  plan 
(TEMP)  covering  Government-related  testing  are  essential  to  TQM.  The 
plans  detail  sufficient  testing  to  prove  conclusively  the  design,  its 
operational  suitability,  and  its  potential  for  required  growth  and  future  utility. 
Test  planning  also  makes  efficient  use  of  test  articles,  test  facilities,  and 
other  resources.  Failure  reporting,  field  feedback,  and  problem  disposition 
are  vital  mechanisms  to  obtaining  a  quality  product. 
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•  Manufacturing  planning  bears  the  same  relationship  to  production  success 
as  test  planning  bears  to  a  successful  test  program  (reference  the 
Manufacturing  Plan  template).  The  overall  acquisition  strategy  includes 
a  manufacturing  strategy  and  a  transition  plan  covering  all  production 
related  activities.  Equal  care  and  emphasis  is  placed  on  proof  of 
manufacture  as  on  proving  the  design  itself.  The  Qualify  Manufacturing 
Process  template  highlights  production  planning,  tooling,  manufacturing 
methods,  facilities,  equipment,  and  personnel.  Extreme  importance  is 
attached  to  subcontractor  and  vendor  selection  and  qualification  including 
flow  down  in  the  use  of  TQM  principles  (reference  the  Subcontractor 
Control  template).  Special  test  equipment,  computer-aided  manufacturing, 
and  other  advanced  equipments  and  statistical  based  methods  are  used 
to  qualify  and  control  the  manufacturing  process. 

TIMELINE 


TQM  oriented  defense  contractors  and  Government  activities  concentrate  on 
designing  and  building  quality  into  their  products  at  the  outset.  Successful  activities 
are  not  content  with  the  status  quo  or  an  acceptable  level  of  quality  approach.  Those 
activities  respond  to  problems  affecting  product  quality  by  changing  the  design  and/or 
the  process,  not  by  increasing  inspection  levels.  Reduction  in  variability  of  the  detail 
design  and  the  manufacturing  process  is  a  central  concept  of  TQM  and  is  beneficial  to 
lower  cost  as  well  as  higher  quality.  Defect  prevention  is  viewed  as  key  to  defect 
control.  Astute  TQM  activities  are  constantly  on  the  alert  to  identify  and  exploit  new 
and  proven  managerial,  engineering,  and  manufacturing  disciplines  and  associate 
techniques. 
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DoD  POSTURE  ON  QUALITY 


•  Quality  is  absolutely  vital  to  our  defense ,  and  requires  a  commitment  to 
continuous  improvement  by  all  DoD  personnel. 

•  A  quality  and  productivity  oriented  Defense  Industry  with  its  underlying  industrial 
base  is  the  key  to  our  ability  to  maintain  a  superior  level  of  readiness. 

•  Sustained  DoD  wide  emphasis  and  concern  with  respect  to  high  quality  and 
productivity  must  be  an  integral  part  of  our  daily  activities. 

•  Quality  improvement  is  a  key  to  productivity  improvement  and  must  be  pursued 
with  the  necessary  resources  to  produce  tangible  benefits. 

•  Technology,  being  one  of  our  greatest  assets,  must  be  widely  used  to  improve 
continuously  the  quality  of  Defense  systems,  equipments  and  services. 

•  Emphasis  must  change  from  relying  on  inspection,  to  designing  and  building 
quality  into  the  process  and  product. 

•  Quality  must  be  a  key  element  of  competition. 

•  Acquisition  strategies  must  include  requirements  for  continuous  improvement  of 
quality  and  reduced  ownership  costs. 

•  Managers  and  personnel  at  all  levels  must  take  responsibility  for  the  quality  of  their 
efforts. 

•  Competent,  dedicated  employees  make  the  greatest  contributions  to  quality  and 
productivity.  They  must  be  recognized  and  rewarded  accordingly. 

•  Quality  concepts  must  be  ingrained  throughout  every  organization  with  the  proper 
training  at  each  level,  starting  with  top  management. 

•  Principles  of  quality  improvement  must  involve  all  personnel  and  products, 
including  the  generation  of  products  in  paper  and  data  form. 
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